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INCREASING  THE  ENLISTMENT  BONUS  CAP  AND  MOS  CHANNELING  EFFECTS 
EXECUTIVE  SUMMARY 


Research  Requirement: 

Faced  with  a  difficult  recruiting  environment,  the  Army  is  moving  to  increase  the  cap  on 
recruiting  bonuses  from  its  current  maximum  of  $20K  to  $40K.  This  policy  change  is  designed 
to  have  MOS  channeling  effects  and  may  also  have  market  expansion  effects.  At  the  top  end, 
this  financial  incentive  targets  recruits  willing  to  enlist  in  high  priority  military  occupational 
specialties  (MOS)  for  relatively  long  enlistment  terms.  Raising  the  bonus  cap  potentially  carries 
significant  implications  for  the  Army’s  ability  to  meet  its  personnel  allocation  requirements.  To 
ensure  that  Army  personnel  managers  and  stakeholders  understand  these  implications  they  need 
to  know  what  impact  raising  the  cap  could  have. 

The  current  research  aimed  to  examine  the  impact  of  raising  the  current  bonus  cap  on  Army 
applicant  job  training  and  term-of-service  (TOS)  choice.  In  particular,  to  what  extent  would 
raising  the  bonus  cap  “channel”  Army  applicants  into  higher  priority  MOS  and  longer  TOS? 


Procedure: 

To  meet  the  study’s  objective,  we  specified,  estimated,  and  applied  a  Job  Choice  Model  (JCM) 
(Diaz,  Ingerick,  &  Sticha,  2004),  which  jointly  models  Army  applicants’  decisions  to  join  or  not 
join  the  Army,  and  their  choices  of  MOS  and  TOS.  A  previous  version  of  this  model  has  proven 
integral  in  current  research  to  evaluate  the  ARI’s  Enlisted  Personnel  Allocation  System  (EPAS). 
Using  actual  applicant  choice  data  from  the  first  quarter  of  FY  2005  ( n  =  18,803),  we  first 
estimated  and  validated  the  JCM.  Once  validated,  we  used  the  JCM  to  simulate  applicants’ 
MOS-TOS  choices  under  two  conditions:  (a)  the  existing  bonus  cap  of  $20K  and  (b)  the 
proposed  bonus  cap  of  $40K. 

Findings: 

Overall,  results  of  our  simulations  indicated  the  following: 

■  Raising  the  current  bonus  cap  to  $40K  is  expected  to  increase  somewhat  overall  Army 
accessions  and  to  uniformly  channel  applicants,  particularly  high  quality  applicants  (i.e., 
I-IIIA ’s  or  those  with  some  college),  to  higher  priority  MOS  and  away  from  low  priority 
ones.  Specifically,  raising  the  cap  is  projected  to  decrease  the  percent  of  non-accessions 
by  0.4%,  while  accessions  to  higher  priority  MOS  are  expected  to  increase,  on  average, 
6.6%.  Among  higher  quality  applicants,  non-accessions  are  expected  to  decline  by  0.5- 
1%;  accessions  to  higher  priority  MOS  are  projected  to  increase,  on  average,  about  8- 
10%;  and  accessions  to  lower  priority  MOS  are  expected  to  decrease  by  2%. 

■  Consistent  with  its  impact  on  accessions,  raising  the  cap  is  expected  to  attract  applicants, 
particularly  higher  quality  applicants,  to  somewhat  longer  TOS  for  higher  priority  MOS. 
Specifically,  TOS  commitments  among  higher  priority  MOS  are  projected  to  increase,  on 
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average,  about  4.0%  for  4  years,  1 1.0%  for  5  years,  and  13.6%  for  6  years,  while 
declining  1.3%  for  3  years.  Similarly,  among  higher  quality  applicants,  TOS 
commitments  are  expected  to  increase,  on  average,  roughly  3.5-5%  for  4  years,  12-16% 
for  5  years,  and  14-17%  for  6  years,  while  declining  1 .5-3.5%  for  3  years— an  increase  in 
the  average  TOS  from  4.10  years  to  4.15  years. 

■  On  average,  raising  the  cap  is  expected  to  increase  the  bonus  dollars  awarded  per 

accession  from  $5,633  to  $6,092 — an  increase  of  8%.  Across  all  applicants,  the  average 
bonus  awarded  to  applicants  accessing  to  higher  priority  MOS  is  projected  to  increase  by 
$1,267.  Overall,  this  translates  into  an  increase  in  the  total  bonus  dollars  awarded  by  the 
Army  from  $77.5  M  to  $83.9  M.  Among  higher  quality  applicants,  the  average  bonus 
dollars  awarded  to  those  accessing  to  higher  priority  MOS  is  expected  to  increase,  on 
average,  by  $l,700-$3,300  — -  a  13-23%  increase. 

Utilization  and  Dissemination  of  Findings: 

These  findings  offer  Anny  personnel  managers  and  stakeholders  insights  into  how  raising  the 
current  bonus  cap  (from  $20K  to  $40K)  could  impact  the  Army’s  ability  to  meet  its  personnel 
requirements.  Operationally,  they  can  be  used  to  guide  future  bonus  policy  and  personnel 
planning.  For  Army  personnel  policy  researchers,  the  methodology,  estimates  and  results  of  Job 
Choice  Modeling  (JCM)  could  be  used  in  future  efforts  to  model  the  impact  of  bonus  policy  on 
Army  applicants’  enlistment  behavior. 
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INCREASING  THE  ENLISTMENT  BONUS  CAP  AND  MOS  CHANNELING  EFFECTS 


Research  Requirement 

The  Army  has  faced  and  continues  to  face  challenges  in  recruiting  youth  to  join  the  Army 
and  in  filling  select  military  occupational  specialties  (MOS),  including  several  that  are  mission 
critical  (GAO,  2005).  In  response  to  this  difficult  recruiting  environment,  the  Army  is  moving  to 
increase  the  cap  on  recruiting  bonuses  from  its  current  maximum  of  $20K  to  $40K.  This  policy 
change  is  designed  to  have  MOS  channeling  effects  and  may  also  have  market  expansion 
effects.1  At  the  top  end,  this  financial  incentive  targets  recruits  willing  to  enlist  in  high  priority 
MOS  for  relatively  long  enlistment  terms.  Raising  the  bonus  cap  potentially  carries  significant 
implications  for  the  Army’s  ability  to  meet  its  personnel  requirements.  To  ensure  that  Army 
personnel  managers  and  stakeholders  understand  these  implications  they  need  to  know  what 
impact  raising  the  cap  could  have. 

The  current  research  aimed  to  examine  the  impact  of  raising  the  current  bonus  cap  on 
Army  applicant  job  training  and  term-of-service  (TOS)  choice.  In  particular,  to  what  extent 
would  raising  the  bonus  cap  “channel”  Army  applicants  into  higher  priority  MOS  and  longer 
TOS? 


To  meet  this  objective,  we  specified,  estimated,  and  applied  a  Job  Choice  Model  (JCM) 
(Diaz,  Ingerick,  &  Sticha,  2004),  which  simulates  Army  applicant  job  choices.  A  previous 
version  of  this  model  has  proven  integral  in  current  research  to  evaluate  the  ARI’s  Enlisted 
Personnel  Allocation  System  (EPAS).  Based  on  actual  applicant  choice  data  from  the  first 
quarter  of  FY  2005,  the  JCM  jointly  models  applicant  decisions  to  join  or  not  join  the  Army,  and 
their  choices  of  MOS  training  and  TOS.  It  produces  probabilities  that  reflect  the  likelihood  of  an 
applicant  choosing  a  particular  MOS-TOS  combination  from  a  list  of  enlistment  alternatives,  as  a 
function  of  his/her  characteristics  and  the  incentives  associated  with  each  alternative.  Once 
estimated  and  validated,  the  JCM  was  employed  to  simulate  applicant  enlistment  decisions  under 
an  increased  bonus  cap  (of  $40K)  to  forecast  its  likely  impact  on  Army  accessions. 

This  report  is  organized  as  follows.  First,  we  provide  an  overview  of  the  JCM  and 
summarize  the  steps  taken  in  estimating  and  applying  it  to  the  current  problem.  Second,  we 
describe  the  design,  conditions,  and  indices  used  to  assess  the  forecasted  impact  of  raising  the 
bonus  cap,  and  related  market  expansion  effects,  on  Army  accessions.  Third,  we  present  findings 
from  this  assessment,  including  estimates  of  the  degree  to  which  the  raised  bonus  cap  will 
channel  Army  applicants  to  higher  priority  MOS  and  longer  TOS.  Finally,  the  report  concludes 
with  a  recap  of  key  findings. 


1  In  Appendix  F  we  extend  the  analysis  to  consider  the  impact  of  the  increased  Enlistment  Bonus  cap  under 
illustrative  market  expansion  rates. 


1 


Procedure 


Modeling  Applicant  Job  Choice: 

Estimation  and  Application  of  the  Job  Choice  Model  (JCM) 

When  making  their  decisions  to  join  the  Army,  applicants  typically  go  through  a  round  of 
processing  at  a  Military  Entrance  Processing  Station  (MEPS).  Upon  successful  completion  of 
this  processing,  the  applicants  then  sit  down  with  an  Army  counselor  to  decide  on  their  MOS 
assignments  and  terms-of-service  (TOS).  Alternatively,  they  may  elect  not  to  join  the  Army. 
Ultimately,  the  applicant’s  enlistment  decision  (i.e.,  to  join  or  not  join  the  Army,  choice  of  MOS, 
and  TOS)  is  influenced  by  many  factors,  including  his/her  characteristics  (e.g.,  gender,  education 
level,  and  ability),  the  types  of  jobs  s/he  is  offered,  and  the  enlistment  incentives  available  for 
each  MOS.  For  example,  male  applicants  are  more  likely,  on  average,  to  enlist  in  electronics  or 
mechanical  jobs  than  female  applicants.  There  are  also  other  factors,  such  as  local  economic 
conditions,  (e.g.,  unemployment  and  poverty  rates)  that  are  known  to  impact  an  applicant’s 
decision.  For  instance,  holding  all  other  factors  constant,  applicants  from  more  difficult 
economic  conditions  (i.e.,  higher  unemployment  and  poverty  rates)  may  be  more  inclined  to 
choose  the  more  highly  incentivized  MOS,  irrespective  of  their  personal  job  preferences. 

To  model  this  decision  process,  the  authors  constructed  and  extended  a  job-choice  model 
(JCM)  that  had  been  developed  previously  to  simulate  Army  applicant  job  choices  (see  Diaz  et 
al.,  2004).  In  the  following  sections,  we  provide  a  conceptual  overview  of  the  JCM,  discuss  its 
advantages,  and  describe  the  steps  taken  to  prepare,  estimate,  and  validate  the  model. 

Overview  of  the  Job  Choice  Model  (JCM) 

The  goal  of  developing  the  JCM  was  to  construct  a  mathematical  model  that  relates 
applicant  characteristics,  attributes  of  their  enlistment  alternatives,  and  local  economic  indicators 
to  their  actual  enlistment  decisions.  Figure  1  provides  a  conceptual  summary  of  the  JCM.  The 
JCM  posits  that  enlistment  decisions  are  a  function  of  applicant  preferences  or  utilities  for 
various  MOS-TOS  enlistment  alternatives.  These  utilities  in  turn  are  a  function  of  (a)  applicant 
characteristics  (e.g.,  demographics,  education,  and  ability),  (b)  attributes  of  enlistment 
alternatives,  and  (c)  local  economic  and  other  indicators.  The  modeling  focuses  on  the  applicant 
decision  making  process  at  the  MEPS.  Other  factors,  such  as  the  role  “influences”  (e.g.,  parents 
and  teachers)  play  on  an  applicant’s  decision,  were  outside  the  scope  of  the  JCM. 
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Figure  1.  JCM  Relating  Applicant  Characteristics,  Alternative  Enlistment  Attributes,  and 
Economic  Indicators  to  the  Enlistment  Decision 


The  JCM  developed  in  this  study  was  based  on  actual  enlistment  choices  made  by  Army 
applicants  during  the  first  quarter  of  FY  2005.  The  enlistment  choices  of  applicants,  MOS 
alternatives,  and  incentives  offered  were  obtained  from  an’ARI  database  containing  REQUEST 
(Recruit  Quota  System)  transactions  between  the  applicant  and  Army  counselor  at  the  MEPS. 
REQUEST  is  the  Army’s  on-line  computer  MOS  training  reservation  system.  The  database 
includes  information  on  qualification  requirements,  such  as  minimum  Aptitude  Area  (AA) 
scores,  gender  restrictions,  driver’s  license,  security  background  check,  and  others.  The  database 
also  reflects  real-world  constraints,  such  as  availability  of  MOS  training  seats.  Generally,  the 
Army  manages  the  incentives  to  target  priority  MOS,  longer  TOS,  and  attract  high  quality 
applicants.  Adjustments  to  the  levels  and  distribution  of  incentives  in  response  to  changes  in 
enlistment  priorities  and  recruiting  trends  are  reflected  in  REQUEST. 

Compared  to  modeling  approaches  based  on  choice  experiments  (e.g.,  conjoint  analysis), 
the  JCM  developed  in  this  study  has  its  advantages  and  disadvantages.  The  biggest  advantage  is 
that  it  reflects  actual  enlistment  behavior  in  a  real  applicant  setting,  in  the  presence  of  a 
counselor  with  priorities  that  may  not  be  consistent  with  the  applicant’s  preferences.  A  second 
advantage  is  that  the  qualification  requirements  and  constraints  reflected  in  the  REQUEST  data 
are  also  carried  over  to  the  JCM.  Having  information  on  these  requirements  is  important  because 
they  impact  the  estimated  relationship  between  incentives  and  enlistment  behavior  and, 
consequently,  the  accuracy  of  forecasted  classification  effects.  This  was  a  major  difficulty 
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encountered  in  recent  research  examining  the  effect  of  enlistment  incentives  on  rating  choice  in 
the  Navy  (Golfin,  2003).  Finally,  because  practically  all  applicant  MEPS  transactions  were 
included  in  the  database,  estimation  and  forecasting  analyses  directly  benefited  from  a  large  and 
representative  sample  of  the  Army’s  applicant  pool.  The  primary  disadvantage  with  the  JCM  is 
that  the  levels  of  various  choice  attributes,  specifically  the  bonus  types  and  amounts,  exhibit  a 
higher  degree  of  correlation  relative  to  the  optimally  designed  attribute-levels  presented  in  choice 
experiments.2  While  potentially  problematic,  variance  restriction  is  less  of  an  issue  when  dealing 
with  the  combined  total  of  bonuses,  which  was  the  focus  in  applying  the  JCM  to  evaluate  the 
increased  bonus  cap.  Additionally,  the  availability  of  a  large  sample  for  estimation  and 
forecasting,  as  was  the  case  here,  potentially  mitigates  the  less-than-optimal  attribute-space 
derived  from  actual  REQUEST  transactions. 

In  sum,  the  JCM  jointly  models  the  applicant  decision  to  join  or  not  join  the  Army,  the 
choice  of  MOS,  and  TOS.  It  produces  probabilities  that  describe  the  likelihood  of  an  applicant 
choosing  a  particular  MOS-TOS  option  from  a  list  of  enlistment  alternatives,  as  a  function  of 
his/her  characteristics  and  the  incentives  available  with  each  alternative.  These  probabilities  were 
then  used  to  analyze  MOS/TOS  channeling  effects  and  to  estimate  increases  in  bonus  costs 
associated  with  raising  the  cap  to  $40K. 

Analysis  Data 

The  analysis  sample  used  in  the  estimation  and  forecasting  application  of  the  JCM 
consisted  of  Anny  applicants  from  the  first  quarter  of  FY  2005.  Specifically,  the  sample  included 
applicants  from  the  ARI  database  with  MEPS  transaction  dates  from  21  September  to  31 
December  2004.  Included  in  the  REQUEST  data  was  detailed  information  describing  the 
transaction  between  applicant  and  counselor,  such  as  applicant  demographics  and  test  scores 
(e.g.,  AFQT  and  AA),  enlistment  alternatives  and  incentives  returned  by  queries  against 
REQUEST  by  the  counselor,  and  the  applicant’s  actual  enlistment  decision.  Enlistment 
alternatives  and  decisions  were  identified  by  MOS  and  reception  station  date.  As  mentioned 
earlier,  the  alternatives  and  incentives  already  reflected  applicant  qualification  requirements  and 
other  MOS  training  availability  constraints.  We  also  supplemented  the  REQUEST  transactions 
data  with  indicators  describing  local  socioeconomic  conditions  from  the  Bureau  of  Labor 
Statistics  and  the  U.S.  Census.  These  indicators  were  merged  to  applicant  records  using  zip 
codes  or  county  information  derived  from  the  applicant  home  address  in  the  database. 

Analysis  variables  used  in  the  JCM  are  summarized  in  Table  1 .  Applicants,  MOS,  and 
TOS  are  indexed  by  i,  m,  and  t,  respectively.  Incentives  indexed  by  both  m  and  t  change  in  level 
by  MOS  and  TOS.  Incentives  indexed  only  by  m  are  constant  across  TOS,  but  are  subject  to 
MOS  eligibility  rules. 


2  While  the  exact  bonus  amount  and  types  offered  change  over  the  course  of  a  fiscal  year  in  response  to  current  and 
emerging  Army  priorities,  these  changes  tend  to  be  comparatively  small. 
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Table  1.  List  of  Alternative  Attributes  and  Applicant  Characteristics  Used  in  the  JCM 


MOS/TOS-Specific  Incentives: 

•  X EB.i.m.t 

Enlistment  Bonus  (EB)  available  to  the  zth  applicant  for  the  wth  MOS  and  t  years  of 
TOS.  This  is  Army’s  primary  monetary  incentive  tool  and  is  offered  in  increasing 
dollar  amounts  by  priority  level  of  an  MOS. 

*  X EB2,i,m,t 

Reduced  amount  of  EB  available  to  the  zth  applicant  for  the  mth  MOS  and  t  years 
of  TOS  when  EB  is  combined  with  ACF. 

Army  College  Fund  (ACF)  available  to  the  zth  applicant  for  the  /wth  MOS  and  t 
years  of  TOS. 

High  Grad  (HG)  bonus  available  to  the  zth  applicant  for  the  mth  MOS.  This  cash 
bonus  is  available  to  applicants  with  varying  levels  of  college  education  (at  least  30 
or  60  college  credit  hours,  and  BA  or  associate  degrees). 

•  XsB.i.m 

Seasonal  Bonus  (SB)  available  to  the  zth  applicant  for  the  mth  MOS.  The  SB 
incentive  is  used  to  encourage  enlistment  into  near  term  training  classes.  It  is 
offered  at  three  levels  depending  on  how  close  training  start  date  is  at  the  time  of 
transaction  at  the  MEPS. 

KZiM 

Airborne  Bonus  (AB)  available  to  the  zth  applicant  for  the  mth  MOS. 

Demographic  Variables: 

Gender  indicator  variable  (l=Male,  0=Female) 

•  ZedC.i 

Indicator  variable  for  education  status  beyond  high  school  graduate  (i.e.,  at  least 
some  college  semester  hours). 

Indicator  variable  for  high  school  graduate  education  status. 

Indicator  variable  for  high  school  senior  education  status. 

Indicator  variable  for  non-high  school  graduate  education  status. 

*  Z/3A.i 

Indicator  variable  for  AFQT  Category  I-IIIIA. 

| 

Score  of  the  applicant  for  the  Aptitude  Area  for  the  wth  MOS. 

Socioeconomic  Indicators: 

Unemployment  rate  in  the  applicant’s  local  county. 

*  Zinc  j 

Median  household  income  in  the  applicant’s  local  county. 

•  Zpoyj 

Poverty  rate  in  in  the  applicant’s  local  county. 

•  Zveli 

Percentage  of  veterans  living  in  the  applicant’s  zip  code  area. 

*  Zrr  i 

Percentage  of  households  in  the  applicant’s  zip  code  area  located  outside  of 
urbanized  area  or  urban  clusters. 

5 


Data  Preparation  and  Simplification 

Because  the  REQUEST  data  represented  real-world  transactions,  certain  features  of  the 
data  were  not  directly  amenable  to  our  modeling  approach  without  simplification  and/or 
transformation.  For  example,  in  the  course  of  a  single  MEPS  transaction,  the  counselor  may 
make  two  or  more  queries  against  REQUEST.  Additionally,  there  were  several  cases  where 
applicants  returned  to  the  MEPS  multiple  times  after  their  initial  transaction.  In  order  to  avoid 
unnecessary  modeling  complexity,  we  made  several  simplifications  to  the  REQUEST  data.  In 
doing  so,  every  effort  was  taken  to  preserve  features  of  the  data  that  are  relevant  in  modeling 
applicant  enlistment  choice  behavior  in  relation  to  incentives. 

Specifically,  there  were  four  features  of  the  REQUEST  data  requiring  attention.  These 
were:  (a)  identifying  applicants’  MEPS  transactions  for  use  in  the  JCM  estimation  and 
forecasting,  as  some  applicants  had  multiple  transaction  dates;  (b)  constructing  a  single  job  list  of 
MOS-TOS  alternatives,  as  many  MEPS  transactions  involved  more  than  one  query  against 
REQUEST  that  likely  returned  duplicate  MOS;  (c)  converting  incentive  codes  recorded  in  the 
REQUEST  data  into  dollar  amounts;  and  (d)  configuring  the  MOS-TOS  enlistment  alternative 
space. 


Identifying  Applicant  Transactions  for  JCM  Estimation  and  Forecasting.  While  a  large 
majority  of  applicants  had  a  single  MEPS  transaction  (about  85%),  there  were  applicants  with 
multiple  transactions.  Of  these,  most  returned  for  a  second  time  (about  11%),  with  the  remaining 
applicants  typically  making  three  to  four  visits.  To  simplify  the  model,  for  those  applicants  who 
returned  to  the  MEPS  multiple  times,  we  retained  their  last  transaction  in  the  database.  We 
elected  to  do  this  as  the  MOS-TOS  alternatives  returned  in  queries  made  during  the  last 
transaction  matched  well  with  actual  enlistment  choices  contained  in  the  reservation  file. 

Constructing  Applicant  Choice  Set.  Enlistment  alternatives  returned  by  REQUEST  are 
indexed  in  the  database  by  MOS  and  reception  station  date.  In  the  course  of  a  single  transaction 
date,  many  applicants  make  multiple  queries  which  can  produce  duplicate  MOS-TOS 
alternatives  with  the  same  or  varying  reception  dates.  To  simplify  this,  we  aggregated 
opportunities  across  queries  within  the  same  transaction  date  and  dropped  duplicate  MOS-TOS 
in  the  combined  list.  This  simplification  essentially  assumes  that  all  alternatives  presented  define 
the  choice  space  for  the  applicant.  Such  an  assumption  is  reasonable  since  in  actual  REQUEST 
transactions  the  applicant  may  decide  at  any  time,  irrespective  of  whether  the  opportunity 
appears  in  the  current  query.  A  slight  complication  occurs  when  duplicate  MOS  have  different 
reception  dates.  Our  analysis  of  the  data  indicated  that  incentives  of  duplicate  MOS  alternatives 
generally  were  identical.  If  differences  existed  between  duplicate  MOS  records,  these  mostly 
concerned  the  availability  and  level  of  the  Seasonal  Bonus  (SB)  incentive.  To  handle  the 
duplicate  MOS  in  the  combined  list,  we  retained  the  (MOS-reception  date)  record  if  the  MOS 
was  the  actual  choice  of  the  applicant,  and  randomly  selected  from  each  set  of  duplicate  records 
associated  with  MOS  not  selected  by  the  applicant.  This  decision  rule  ensures  that  the  JCM 
always  included  incentives  that  factored  into  the  applicant’s  final  enlistment  decision  without 
systematically  biasing  their  impact.3 


3  For  example,  always  keeping  the  earliest  reception  date  would  likely  lead  to  underestimation  of  the  impact  of  the 
SB  incentive. 
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Converting  Incentive  Codes  to  Dollars.  While  the  REQUEST  data  records  the  codes 
representing  the  different  types  of  incentives  available  for  an  MOS  for  a  given  applicant,  the 
actual  dollar  value  associated  with  an  incentive  must  be  “computed”  as  a  function  of  the  MOS 
and  transaction  date4.  With  the  exception  of  the  SB  incentive,  information  on  the  dollar  values 
associated  with  different  incentives  could  be  found  in  MILPER  messages  that  specify  the 
Army’s  existing  bonus  policy  and  incentive  structure.  Using  the  MILPER  messages  covering  the 
first  quarter  of  FY  2005,  we  converted  the  incentive  codes  in  the  database  to  their  corresponding 
dollar  values.  In  the  case  of  the  SB  incentive,  information  in  the  MILPER  messages  was  not 
specific  beyond  “as  needed.”  Additionally,  our  analysis  of  the  SB  amounts  in  the  reservation 
records  of  applicants  did  not  exactly  match  the  incentive  codes  by  SB  priority  levels.  We 
therefore  conducted  descriptive  and  tabulation  analyses  to  estimate  a  reasonable  dollar  amount 
by  SB  priority  level  and  MOS. 

Configuring  the  Enlistment  Alternative  Space.  Inspection  of  applicants’  MOS  choices 
indicated  a  total  of  150  MOS.  When  combined  with  TOS,  this  yielded  a  considerably  large 
choice  space,  which  was  problematic  for  computational  and  model  accuracy  purposes.  To 
address  this,  we  reduced  the  choice  space  as  follows.  First,  we  combined  small  MOS  that  were 
similar  in  job  content  (i.e.,  belonging  to  the  same  Career  Management  Field)  and  had  similar 
incentive  levels.  Second,  to  obtain  a  JCM  that  is  relevant  to  the  research  requirements  of  this 
study,  we  kept  MOS  with  priority  levels  1  or  2  during  the  first  quarter  of  FY  2005  as  singletons. 
The  Army  uses  nine  levels  to  prioritize  MOS,  with  level  1  as  the  top  priority  and  level  9  as  the 
lowest  priority.  Finally,  for  the  remaining  MOS,  we  aggregated  those  with  smaller  n  and  similar 
job  content  based  on  their  priority  levels.  For  this  final  aggregation,  we  grouped  the  remaining 
priority  levels  into  three  categories:  (a)  priority  levels  3  and  4;  (b)  priority  levels  5  through  7;  and 
(c)  priority  levels  8  and  9.  After  combining  MOS,  there  were  a  total  of  36  “job  alternatives.” 
Henceforth,  we  will  continue  to  refer  to  the  36  reconfigured  job  alternatives  as  MOS  alternatives. 
In  combination  with  the  TOS  dimension,  this  reconfiguration  produced  a  choice  space  with  136 
alternatives,  including  the  alternative  of  not  joining  the  Army.  This  was  further  reduced  to  104 
alternatives  after  dropping  27  MOS-TOS  alternatives  whose  combined  total  accounted  for  less 
than  0.3  percent  of  the  sample.  The  final  set  of  36  MOS  alternatives  is  summarized  in  Table  2. 


4  One  must  take  into  account  transaction  date,  as  there  are  periodic  changes  in  the  dollar  values  associated  with 
different  incentives. 
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Table  2.  MOS  Alternative  Configuration  and  Clusters 


Alternative 

ID  Label 

Cluster* 

Reduced 

Cluster 

MOS 

1 

1 IX 

1 

1 

1  IB, 11C, 11X 

2 

13F 

2 

1 

13F 

3 

FA1 

2 

1 

13B,  13D,  13M 

4 

FA2 

2 

1 

13P,  13R,  13S,  13W 

5 

ADI 

2 

1 

14E,  14  J,  14R,  14S,  14T 

6 

AVI 

7 

4 

15B,  15D,  15F,  15G,  15H,  15J,  15N,  15R,  15S,  15T,  15U,  15Y 

7 

AV2 

7 

4 

15P,  15Q 

8 

18X 

1 

1 

18X 

9 

19D 

1 

1 

19D 

10 

19K 

1 

1 

19K 

2 ID,  21E,  21F,  21J,  21K,  21L,  21M,  21R,  21 S,  21T,  21U,  21V, 

11 

EN1 

10 

7 

21 W,  44B,  44E,  62B 

12 

EN2 

10 

7 

21B,  21C 

13 

SI1 

14 

10 

25B,  25D,  25M,  25R,  25V 

14 

SI2 

3 

2 

25C,  25F,  25L,  25P,  25Q,  25S,  25U,  31C,  31R 

15 

PA1 

4 

3 

37F,  46Q,  46R 

16 

LEI 

4 

3 

31B, 31E 

35A,  35D,  35F,  35H,  35K,  35L,  35M,  35P,  35R,  35T,  35Y,  52C, 

17 

ELI 

6 

4 

91 A 

18 

EL2 

6 

4 

33W,  35E 

19 

AX1 

8 

5 

27D,  42A,  42L,  44C,  56M 

20 

AMI 

11 

4 

45B,  45G,  45K 

21 

52D 

5 

4 

52D 

22 

63B 

5 

4 

63B 

23 

VM1 

5 

4 

63D,  63H,  63J,  63M 

24 

VM2 

5 

4 

63A 

25 

74D 

13 

9 

74D 

26 

TR1 

9 

6 

88H,  88K,  88L,  88N 

27 

88M 

9 

6 

88M 

28 

89D 

4 

3 

89D 

29 

89B 

9 

6 

89B 

91D,  91E,  91G,  91H,  91J,  91K,  91M,  91P,  91Q,  91R,  91S,  91T, 

30 

MD1 

12 

8 

91V,  91W,  91X 

31 

92F 

9 

6 

92F 

32 

92G 

9 

6 

92G 

33 

SL1 

9 

6 

92A,  92L,  92M,  92R,  92S,  92W,  92Y 

34 

INI 

3 

2 

96B,  96D,  96U,  98J,  98K 

35 

HI1 

5 

3 

97B, 97E 

36 

98X 

3 

2 

98X 

*  MOS-Clustcr  Titles:  l=Closc  Combat;  2=Non  Linc-of-Sight  Fire;  3=Surveillancc,  Intelligence,  and  Communications;  4=Security 
and  Civil  Affairs;  5=Mechanical  Maintenance  Repair;  6=  Electronics  Maintenance  Repair;  7=Aircraft  Maintenance  Repair; 
8=Administration;  9=Logistics/Supply  Support;  10=Hcavy  Equipment  Operator;  1 1  =Craftworker;  12=Medical  Care,  Health,  and 
Well-Being;  13=Skilled  Science  Technician;  14=Media  Specialist 
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JCM  Estimation 


To  model  applicants’  enlistment  decision  process,  we  used  discrete  choice  methodology. 
This  methodology  is  commonly  used  in  econometrics  and  marketing  to  model  the  choice 
behavior  of  an  individual  selecting  from  a  finite  set  of  alternatives  (Train,  1986;  Ben-Akiva  & 
Lerman,  1985;  Greene  2000).  Using  this  methodology  enabled  us  to  relate  the  probability  of  an 
applicant  choosing  an  enlistment  alternative  to  the  attributes  of  that  alternative  and  the 
characteristics  of  the  applicant.  Given  the  requirements  of  this  study,  it  was  important  to  develop 
a  JCM  that  jointly  models  the  applicant’s  decision  to  join  or  not  join  the  Army,  choice  of  MOS, 
and  TOS.  In  this  way,  important  aspects  of  Army  enlistment  incentive  policy  that  simultaneously 
entice  enlistments  in  priority  MOS  and  longer  TOS  can  be  more  accurately  represented  in  the 
JCM.  Our  development  of  the  JCM  emphasized  systematic  relationships  that  are  useful  for 
predicting  or  forecasting  enlistment  choices  of  applicants.  The  specification  of  this  model,  results 
of  the  estimation,  and  diagnostics  are  discussed  in  the  following  sections. 

Specification  of  the  Model.  Typically  discrete  choice  modeling  is  based  on  the 
multinomial  logit  (MNL)  model.  While  informative,  there  are  important  behavioral  aspects  of  the 
Army  applicant  choice  process  that  were  not  supported  a  priori  by  the  assumptions  of  this  model. 
For  instance,  the  MNL  assumes  independence  from  irrelevant  alternatives,  which  holds  that  an 
individual’s  relative  preference  for  two  alternatives  is  not  affected  by  the  availability  of  a  third 
alternative.  This  assumption  is  not  likely  to  hold  in  the  current  choice  context  given  the  similar 
types  of  jobs  applicants  are  presented,  even  under  the  reduced  alternative  choice  space.  For 
example,  an  applicant  would  likely  view  combat  jobs  such  as  MOS  1 IX  (infantry)  and  13F  (field 
artillery)  to  be  more  similar  to  one  another  than  to  administration  jobs  such  as  MOS  42A 
(Human  Resource  Specialist)  or  44C  (Accounting  Specialist).  Therefore  the  preference  for  MOS 
1 IX  relative  to  42A  will  decrease  if  13F  is  also  available  to  the  applicant.  To  address  this,  we 
employed  a  more  flexible  framework  with  less  restrictive  assumptions,  the  mixed  multinomial 
logit  (MMNL)  model,  to  develop  the  JCM. 

A  first  step  in  developing  the  JCM  is  to  mathematically  specify  the  utility  (or  value)  that 
an  applicant  places  on  an  alternative.  As  to  be  expected,  an  applicant’s  utility  for  a  given 
enlistment  alternative  would  likely  be  derived  from  that  alternative’s  attributes,  such  as  type  of 
job  (MOS),  length  of  commitment  (TOS),  and  various  monetary  incentives.  For  example, 
increasing  the  bonus  for  a  specific  enlistment  alternative  would  likely  increase  its  total  utility  and 
therefore  would  improve  its  probability  of  selection.  Because  they  are  based  on  individual 
perceptions,  utilities  are  also  expected  to  differ  across  individual  applicants  even  when  attributes 
of  alternatives  are  held  constant.  To  some  extent,  these  individual  differences  can  be  explained 
by  observable  applicant  characteristics  (e.g.,  gender,  education  level,  and  ability). 

As  with  any  empirically  based  model,  there  will  be  residual  utilities  that  cannot  be 
explained  by  observable  data.  In  our  specification,  we  partitioned  this  residual  utility  into  two 
components.  The  first  component  reflected  unobserved  applicant  characteristics  not  directly 
captured  by  our  data  that  would  be  related  to  and  predictive  of  enlistment  choices  (e.g.,  applicant 
motivation,  vocational  interests,  and  so  on).  The  second  component  represented  the  usual 
disturbance  term  (i.e.,  random  error). 
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To  formally  summarize  the  above  description,  we  denote  the  utility  to  the  z'th  applicant  of 
the  enlistment  alternative  corresponding  to  the  mth  MOS  and  rth  TOS  by: 


vIM=rM(x,z)+Flf,  +  EM 


The  tenn  Vi  m ,  (X,  Z)  represents  the  systematic  component  of  utility,  which  is  linked  in 

predictable  ways  to  the  attributes  of  the  (m,t)  alternative  and  characteristics  of  the  ith  applicant. 
The  residual  component  is  the  sum  Fi  m  +  Ei  m  ( ,  where  Fi  m  represents  error  related  to  unobserved 

characteristics  of  the  applicant  and  Ei  ml  is  pure  disturbance  (i.e.,  random  error).  From  the 

researcher’s  viewpoint,  these  two  error  components  also  represent  sources  of  uncertainty  about 
the  applicant’s  choice  behavior.  The  forms  of  the  systematic  and  residual  utilities  are  specified 
later  below. 


A  second  step  in  the  specification  of  the  JCM  is  specifying  the  decision  protocol  used  by 
the  applicant  when  choosing  among  the  set  of  available  enlistment  alternatives.  Consistent  with 
conventional  applications  of  discrete  choice  modeling,  we  developed  the  JCM  based  on  the 
random  utility  maximization  (RUM)  assumption.  The  RUM  posits  that  among  all  available 
alternatives,  an  individual  will  choose  the  alternative  that  has  maximum  value  to  him/her. 5  In  the 
current  context,  this  means  that  in  choosing  among  enlistment  alternatives,  applicants  are 
expected  to  select  those  enlistment  alternatives  with  the  greatest  utility.  More  formally,  letting 
E-ml  =  Fjm  +  Eimt ,  the  probability  that  the  ith  applicant  chooses  the  alternative  ( m',t ')  is 
described  by: 


P,(m\r)=  Pfi and<V,} 

=  Pr{  V„.v  ~  K.m.,  >  K..f  -  K.y ;  rri*  rn  and  ('  *  l  j 

The  first  equality  simply  restates  the  RUM  assumption.  The  second  equality  states  that  the 
systematic  utility  of  the  chosen  alternative  exceeds  the  systematic  utilities  of  all  not  chosen 
alternatives  by  an  amount  that  is  larger  than  the  differences  between  their  residual  utilities.  In 
other  words,  the  probability  of  an  applicant  selecting  a  specific  alternative  increases  as  the 
systematic  utility  of  that  alternative  for  the  applicant  increases  relative  to  the  other  alternatives. 
Additionally,  the  last  equality  indicates  that  if  the  systematic  utilities  are  not  strong  (i.e.,  random 
component  dominates  the  random  utility),  the  choice  probabilities  from  the  model  would  not  be 
different  from  those  obtained  from  random  selection. 

Specifying  the  Systematic  Utility.  We  now  fully  express  the  systematic  utility  in  terms  of 
monetary  incentives,  applicant  demographics,  and  aptitude  scores  obtained  from  the  REQUEST 
transaction  data.  In  the  following  we  use  the  indices  /,  m,  and  t,  in  the  utility  expression  to 
indicate  that  a  parameter  or  variable  varies  by  applicant,  MOS,  and  TOS,  respectively.  For  the 


5  Recently,  generalizations  of  conventional  discrete  choice  modeling  that  relax  the  RUM  assumption  have  been 
proposed.  One  such  approach  calls  for  explicitly  incorporating  motivational  and  attitudinal  factors  when  modeling 
individuals’  choices  (Walker,  2001). 
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enlistment  alternative  associated  with  the  mth  MOS  and  t  years  of  TOS,  the  full  expression  for 
the  systematic  utility  is  given  by: 

K,mj  (x  ■>  Z)  =  AM  m  +  GsexM  mZsexM  j  +  Gl3A  mZUAi  +  G edGC  mZ edGC +  GAAZAA  j  m 
+  ATt  +  H  sexMtZsexM  t  +  HnAlZxiAi  +  GedGClZedGCi 

BhqX HG,i,m  "l~  BSBXsB,i,m  B  AB X  ARim 

+  BEbX +  ^B EBdsX EBdsimt  +  B  EBdmX EBdmimt )  +  B  ACX  ACiml 

+  Bbc  max(o,  XTB  j  m  t  —  Ct ) 


for  the  103  ( m,t )  combinations  included  in  the  reduced  enlistment  alternative  space,  m=  1  ,...,36 
and  t=  3,. ..,6.  The  ^-constants,  G-,  H-,  and  5-coefficients  are  parameters  to  be  estimated  from  the 
data.  Characteristics  of  the  ith  applicant  are  represented  by  Z  variables  while  the  attributes  of  the 
( m,t )  enlistment  alternative  are  represented  by X variables.  Except  for  XEBds jmt,  XEBdns iml,  and 

X TBim.t  >  which  will  be  described  below,  all  other  variables  were  defined  previously  in  Table  1. 

The  first  two  lines  include  MOS-  and  TOS-specific  constants  and  the  component  of 
systematic  utility  related  to  applicant  demographics  (i.e.,  gender,  high  school  degree  status,  and 
AFQT  I-IIIA  Category)  and  AA  scores.  To  obtain  a  parsimonious  model,  interactions  between 
applicant  demographics  and  enlistment  alternatives  were  constructed  separately  with  respect  to 
the  MOS  and  TOS  dimensions  of  the  enlistment  alternatives,  as  opposed  to  directly  specifying 
interactions  relative  to  the  103  MOS-TOS  combinations  in  the  choice  space.  Furthermore, 
interactions  between  applicant  demographics  and  MOS  were  restricted  to  be  equal  within  an 
MOS-cluster,  a  group  of  MOS  based  on  job  requirements/description.  The  14  MOS-clusters  used 
in  the  JCM  are  identified  under  the  column  “Cluster”  in  Table  2.  The  third  line  in  the  utility 
expression  represents  a  component  of  systematic  utility  explained  by  monetary  incentives  that 
can  vary  across  the  MOS-dimension  of  the  alternative  space  (i.e.,  high  grad  bonus,  seasonal 
bonus,  and  Airborne  bonus). 

The  fourth  line  in  the  utility  expression  represents  the  component  of  utility  explained  by 
the  EB/ACF  incentive.  Unlike  the  first  three  lines  in  the  utility  expression,  this  component  varies 
with  respect  to  both  MOS  and  TOS  dimensions  of  the  alternative  space.  The  formulation  of  this 
component  was  complicated  by  the  two  optional  ways  the  EB/ACF  incentive  is  offered  to 
applicants:  as  a  full  cash  bonus  amount  or  a  reduced  cash  bonus  plus  college  money,  if  both 
options  are  available.  The  reduction  in  the  full  EB  dollar  amount  (i.e.,  XEBimt)  is  represented  in 

the  utility  expression  by  the  variables  XEBds  i  m  t  and  XEBdm  i  m  t ,  for  senior  and  non-senior 

•  •  A  •  ^  if 

applicants,  respectively.  The  coefficients  BEBcLs  and  BEBdns  were  specified  to  be  normally 


These  variables  are  constructed  as  the  interactions  between  the  senior  category  indicator  variable  and  the 

=  t1  “  ZedS,i  )X  XEBd,i,m,t  >  where 

is  the  reduction  in  the  full  EB  amount. 


reduction  in  EB  amount:  XEBds  i  m  t  =  ZedSJ  X  XEBdJ  md  and  XEBdns  i  m  l 


XEBd,i,m,t  —  XEB,i,m,t 


X 


EB2,i,m,t 


n 


distributed  random  parameters  with  different  means  and  a  common  variance.  We  used  random 
coefficient  parameters  as  a  mechanism  for  combining  the  two  options  in  a  single  utility 
expression,  and  to  avoid  the  need  to  explicitly  model  applicant’s  choice  between  EB-only  and 
EB+ACF  alternative  forms  of  the  EB/ACF  incentive.  Specifying  two  random  coefficients  with 
different  means  allows  senior  and  non-senior  applicants  to  differ  in  their  average  preference  for 
the  two  forms  of  the  EB/ACF  incentive.7 8 

The  last  line  in  the  utility  expression  represents  an  adjustment  term  to  keep  the  systematic 
utility  derived  from  bonus  incentives  behaviorally  consistent  with  the  choice  situation  faced  by 
the  applicant.  The  quantity  max(o,  XTB  im  (  -C,)  represents  the  amount  of  the  total  bonus  for  the 

( m,t )  enlistment  alternative  that  exceeds  the  bonus  cap.  The  variable  XTB  i  m  l  in  this  quantity  is 

the  pre-cap  total  bonus  XHGim  +  XSBim  +  X  ABim  +  XEB  i  m  t  and  the  constant  C,  equals  the  bonus 

cap  for  t  years  of  TOS.  Without  this  adjustment  term  in  the  systematic  utility,  for  example,  a 
$30K  total  bonus  would  be  interpreted  as  having  twice  the  value  of  a  $1 5K  total  bonus. 
Technically,  this  is  not  possible  because  the  applicant  can  only  receive  $20K  of  the  $30K  total 
bonus  amount  (under  the  extant  bonus  cap  policy).  This  term  is  the  key  in  applying  the  estimated 
JCM  to  forecast  the  effects  of  increasing  the  bonus  cap  from  $20K  to  $40K  in  this  evaluation 
study. 


For  the  alternative  corresponding  to  the  decision  not  to  join  the  Army,  we  specified  a 
linear  systematic  utility  by  combining  applicant  demographics  from  REQUEST  data  and 
socioeconomic  variables  from  the  secondary  data  sources.  The  full  expression  is  shown  below. 

In  this  case,  the  applicant  education  status  in  the  non-accession  utility  formulation  was  expanded 
to  four  categories  (i.e.,  not  HS  Graduate,  HS  Senior,  HS  Graduate,  College+). 


^,999,o(^>^)  =  A/ ,999  +  GsexM  ,999^ sexM  ,i  +  ^13/1,999^13/),/ 

+  ^  edN  .999^  edN  ,i  +  ^  edG  .999^  edG  ,i  +  ^  edC  ,999^  edC  ,i 

+  ^uem,999^emp,i  +  ^  pov.999^  pov.i  +  ^  inc,999^incj  +  ^  ve/,999  ^ve/,/  +  ^rr,999^rrj 


Specifying  the  Residual  Utility.  In  addition  to  specifying  the  systematic  utility,  we  needed 
to  specify  the  distributional  structure  of  the  residual  utility.  Earlier  we  expressed  the  residual 
utility  as  Fi  m  +  Ej  m , ,  where  F.  m  represents  error  related  to  unobserved  applicant  characteristics 

and  EimJ  represents  pure  disturbance  (i.e.,  random  error).  In  specifying  the  distributional  form 
for  Fi  m ,  we  first  noted  that  unobserved  applicant  characteristics  would  most  likely  explain 


7  This  formulation  was  motivated  by  viewing  EB-only  dollar  amount  as  representative  of  the  full  value  of  the 
incentive  and  the  EB+ACF  option  as  an  alternative  package  with  approximately  equal  value.  From  this  standpoint, 

the  term  B  EB(hX  EBds  jml  +  B AC X AC ;  m ,  can  be  interpreted  as  the  net  effect  of  reducing  the  cash  bonus  and  adding 
college  money  in  the  incentive  package  for  a  senior  applicant. 

8  We  conducted  plausibility  checks  by  comparing  the  actual  total  bonus  received  by  applicants  who  accessed  (as 
reported  in  the  REQUEST  data)  with  the  corresponding  capped  total  bonus  given  by  min(  C, ,  XTB ;  m  t ).  Our 
analysis  showed  high  correlation  (95%)  between  the  actual  total  bonus  and  computed  total  bonus. 
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shared  utility  across  similar  jobs  or  MOS.9  To  put  this  postulated  behavior  in  the  model,  we 
specified  Fi  m  to  be  a  random  utility  component  that  is  common  to  all  MOS  within  an  MOS- 
cluster,  for  all  MOS-TOS  (i.e.,  accession)  enlistment  alternatives,  having  a  normal  distribution 
with  zero  mean  and  standard  deviation  (SD)  to  be  estimated  from  the  data.  The  Fi  m  s  were 
assumed  to  be  uncorrelated  across  MOS-cluster  for  a  given  applicant.  In  contrast,  the  disturbance 
terms  E,  m ,  ’s  were  assumed  to  have  the  standard  Gumbel  distribution  with  zero  mean, 
independently  distributed  across  all  (accession  and  non-accession)  enlistment  alternatives. 
Finally,  we  also  specified  an  additional  random  utility  component,  Fj  m  ,  in  the  non-accession 

utility  of  applicants  who  were  offered  multiple  MOS  alternatives.  This  extra  component  reflects 
heterogeneity  in  residual  utility  for  the  non-accession  alternative  (or  equivalently,  for  joining  the 
Army)  between  applicants  offered  single  MOS  and  those  offered  multiple  MOS  alternatives. 

This  specification  posits  a  smaller  variance  for  applicants  choosing  between  not  joining  the 
Army  and  a  single  MOS. 

Estimation  Results.  We  estimated  the  JCM  parameters  using  the  mixed  logit  estimation 
procedure  in  the  BIOGEME  software  (Bierlaire,  2003).  The  procedure  employs  a  maximum 
simulated  likelihood  method  for  estimating  model  parameters.  Included  in  this  method  is  a 
simulation-based  approach  for  approximating  the  multidimensional  distribution  of  unobserved 
applicant  characteristics  to  evaluate  the  likelihood  during  estimation.10  To  further  simplify  model 
estimation,  we  constrained  the  Fims  of  MOS-clusters  that  are  similar  to  be  equal,  reducing  the 

dimension  of  the  unobserved  applicant  characteristics  used  for  modeling  correlation  among 
alternatives  from  14  to  10.  Table  2  shows  this  smaller  MOS-cluster  configuration  under  the 
column  “Reduced  Cluster.”  Altogether,  the  estimation  involved  a  12-dimensional  multivariate 
normal  distribution  for  each  applicant:  (a)  10  dimensions  for  applicant  characteristics  (  Fj  m, , 

m  -1,...,10),  (b)  one  for  the  additional  error  term  for  applicants  facing  multiple  MOS  (IE  999  ),  and 

(c)  one  for  the  random  coefficient  for  the  reduced  EB  dollar  amount  (  B*Egds  or  B*EBdns ).  We  used 

three  hundred  Halton  draws  (quasi  random  numbers)  from  this  distribution  during  the  estimation 
for  each  applicant. 

Given  the  very  large  analysis  data  set  available  from  the  REQUEST  data,  we  employed  a 
sampling  procedure  to  identify  a  subset  of  applicants  for  JCM  estimation.  The  remaining 
applicants  were  used  for  out-of-sample  model  fit  diagnostics.  Out  of  a  total  of  18,803  applicants, 
4,020  were  used  in  the  estimation,  and  the  remaining  14,783  were  used  for  out-of-sample 
diagnostics.  To  identify  the  estimation  sample,  we  employed  a  choice-based  sampling  to  ensure 
that  all  MOS-TOS  alternatives  were  adequately  represented  during  estimation.  This  approach 
avoided  over-representation  of  a  few  alternatives,  such  as  those  associated  with  1 IX  and/or  TOS 
of  four  years,  that  would  be  obtained  if  simple  random  sampling  was  used.  To  carry  out  the 
choice-based  sampling,  we  first  grouped  applicants  according  to  their  chosen  MOS-TOS 


9  Readers  are  reminded  that  this  is  one  of  the  main  motivations  for  using  a  mixed  multinomial  logit  model 
(MMLM).  For  a  given  applicant,  the  utilities  of  MOS  belonging  to  the  same  cluster  can  be  expected  to  share  a 
residual  utility  and  thus  be  correlated.  Not  taking  into  account  this  correlation  would  seriously  violate  the 
independence  from  irrelevant  alternatives  (II A)  assumption  in  the  standard  MNL  choice  model. 

10  For  details  on  the  estimation  method,  see  Train  (2003). 
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combination  (or  non-accession  choice).  Applicants  were  then  selected  by  under-sampling  from 
the  larger  MOS-TOS  groups  and  over-sampling  from  the  smaller  MOS-TOS  groups.  To  obtain 
unbiased  JCM  estimates,  we  assigned  weights  to  applicants  during  the  estimation  that  were  equal 
to  the  reciprocal  of  the  sampling  rates  in  their  respective  MOS-TOS  groups. 

The  JCM  parameter  estimates  obtained  at  the  end  of  estimation  runs  are  presented  in 
Table  3,  along  with  their  corresponding  standard  errors  (S.E.)  and  t-statistics.  To  protect  against 
model  misspecification  error,  standard  errors  and  t-statistics  are  based  on  robust  variance- 
covariance  matrix  estimates  (Bierlaire,  2003).  Bolded  t-statistics  are  significant  at  the  .05  level. 
Where  standard  error  and  t-statistic  values  are  blank,  the  corresponding  parameters  were  fixed  at 
zero.11 


11  Parameters  were  fixed  to  zero  to  either  render  the  model  identifiable,  or  in  some  cases,  to  be  consistent  with 
official  Army  policy  (e.g.,  females  are  restricted  from  combat  jobs,  thus  gender  by  MOS-cluster  interactions 
involving  MOS  with  female  restrictions  were  fixed  at  zero). 
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Table  3.  JCM  Parameter  Estimates 


Parameter 

Estimate 

S.E. 

t-stat 

MOS-Specific  Constants 

Am.i 

0.0000 

Am.2 

-0.7624 

0.2372 

-3.21 

Am.3 

-1.1231 

0.2342 

-4.79 

Am.4 

-2.5115 

0.3067 

-8.19 

Am, 5 

-2.6821 

0.2878 

-9.32 

Am, 6 

1.2892 

0.8200 

1.57 

Amj 

-1.2066 

0.9239 

-1.31 

Am, 8 

3.5881 

0.7161 

5.01 

Am.9 

-1.0108 

0.1369 

-139 

Am.  io 

-1.6915 

0.1750 

-9.67 

Am.h 

-2.5045 

0.7158 

-3.50 

Am.  12 

-2.8826 

0.7543 

-3.82 

'll/./! 

-0.0574 

0.7110 

-0.08 

Am.  14 

-2.5957 

0.6909 

-3.76 

Am.  is 

-2.5129 

0.8303 

-3.03 

Am.  16 

-3.0446 

0.8238 

-3.70 

Am.I  7 

-4.4955 

0.7488 

-6.00 

Am.  is 

-4.0014 

0.8487 

-4.71 

Am.  19 

-6.5450 

3.6689 

-1.78 

Am.20 

-3.5432 

0.7305 

-4.85 

Am,2I 

-2.9522 

0.6658 

-4.43 

Am.  22 

-1.3146 

0.6238 

-2.11 

Am.23 

-2.7946 

0.6423 

-4.35 

Am. 24 

-2.7284 

0.6553 

-4.16 

Am.23 

-0.9564 

0.5852 

-1.63 

Am. 26 

-1.1420 

0.4707 

-2.43 

Am. 2? 

-0.9464 

0.4800 

-1.97 

Am.  28 

-4.3761 

0.8746 

-5.00 

Am. 29 

-1.6117 

0.4948 

-3.26 

Am,30 

-2.5487 

0.8940 

-2.85 

Am. 31 

-0.2838 

0.4585 

-0.62 

Am.32 

-0.8950 

0.4811 

-1.86 

Am.  3  3 

-0.9678 

0.4782 

-2.02 

Am.  34 

-2.3710 

0.6993 

-3.39 

Am.  35 

-2.9897 

0,7566 

-3.95 

Am. 36 

1.9769 

1.1299 

1.75 

Am.999  2.5304 

TOS-Specific  Constants 

2.2607 

1.12 

Atj 

0.0000 

Atm 

-1.3196 

0.1640 

-8.05 

A  T.5 

-2.3094 

0.2273 

-10.16 

A  T.6 

-1.7773 

0.4624 

-3.84 
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Table  3.  JCM  Parameter  Estimates  (con ’t) 


Parameter 

Estimate 

S.E. 

t-stat 

MOS  AA  Score  and  Incentives 

GAa 

0.7768 

0.2252 

3.45 

Bug 

0.1875 

0.0509 

3.69 

B.SH 

-0.0015 

0.0143 

-0.10 

Bab 

0.4232 

0.0531 

7.97 

Bek 

0.0551 

0.0152 

3.63 

B  Kirns 

0.0653 

0.0488 

1.34 

Bebdn 

-0.0547 

0.0354 

-1.54 

Bag 

0.0228 

0.0038 

6.01 

Bbc 

-0.1561 

0.0327 

-4.77 

MOS  Cluster-Subgroup  Interactions 

G/3A.1 

-1.0648 

0.4207 

-2.53 

Giu.: 

-0.8062 

0.4162 

-1.94 

G/3A.3 

0.0000 

G/3A.S 

0.3177 

0.4589 

0.69 

Gl3J.« 

-1.3475 

0.4107 

-3.28 

G/3A.7 

-0.6643 

0.4966 

-1.34 

G  I3A.S 

-0.4673 

0.6302 

-0.74 

G I3A  9 

2.2689 

1.1442 

1.98 

G  13A.I0 

-1.1306 

0.3990 

-2.83 

Gl3A.ll 

-0.7511 

0.4051 

-1.85 

Gi3a.i : 

-0.1560 

0.5056 

-0.31 

G 13A.  13 

2.1035 

0.7725 

2.72 

G  ISA.  n 

-0.9956 

0.4674 

-2.13 

G/3A.IS 

-0.2312 

0.4940 

-0.47 

G/ssm 

1.4763 

0.5437 

2.72 

G'dGC.I 

-0.3072 

0.3379 

-0.91 

GcdCC.3 

-0.615J 

0.3334 

-1.85 

GdCC.3 

0.0000 

G dec.  3 

0.8419 

0.3908 

2.15 

G,dCC.6 

-0.2796 

0.3279 

-0.85 

GedGC.7 

0.1703 

0.4758 

0.36 

G'dcc.x 

0.3912 

0.4320 

0.91 

GedGC.9 

1.3107 

0.9677 

1.35 

GedGC,  10 

0.1167 

0.3007 

0.39 

Gedccn 

0.1986 

0.3586 

0.55 

GedGC.  12 

-0.4365 

0.4146 

-1.05 

GedGC.  13 

0.6251 

0.4157 

1.50 

GrdGC.  U 

0.5800 

0.4270 

1.36 

G'dcc.ts 

0.2014 

0.4363 

0.46 

Gedc.m 

-0.3850 

0.6056 

-0.64 

GedG.999 

-3.3015 

0.5489 

-6.01 

GedS.999 

0.0000 

GedN.999 

-3.8863 

0.6589 

-5.90 
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Table  3.  JCM  Parameter  Estimates  (con ’t) 


Parameter 

Estimate 

S.E. 

t-stat 

GsexM.t 

0.0000 

Q$exM,2 

0.0000 

GgexM.3 

0.0000 

GsexM.5 

-0.7348 

0.4222 

-1.74 

G sexM.6 

0.7846 

0.4471 

1.75 

GsexM.  7 

1.5789 

0.5547 

2.85 

GsexM.  8 

0.0190 

0.4866 

0.04 

GsexM.9 

-0.5292 

1.0841 

-0.49 

(-JggxM,  10 

-1.1281 

0.3553 

-3.17 

GsexM,  1 1 

0.6076 

0.4537 

1.34 

G sexM 12 

0.5535 

0.5119 

1.08 

G sex  M.  13 

-1.3681 

0.4723 

-2.90 

GsexM.  14 

-1.4622 

0.4196 

-3.48 

GsexM.ll 

-1.1363 

0.4584 

-2.48 

G sex  M.  999 

-2.3366 

0.5569 

-4.20 

TOS-Subgroup  Interactions 

Hjja.s 

0.0000 

Umax 

1.8759 

0.1157 

16.22 

Hua.s 

1.9189 

0.1814 

10.58 

H  Ha. l 

1.4996 

0.4272 

3.51 

H«:gc.3 

0.0000 

HedGC.4 

0.6823 

0.1077 

6.33 

HetICC.i 

0.6888 

0.1379 

5.00 

HctIGCS S 

0.2457 

0.2151 

1.14 

HsexM.3 

0.0000 

HsexM.4 

-1.0342 

0.1317 

-7.85 

HgexM.S 

-1.0169 

0.1705 

-5.96 

HsexM.  6 

-1.4311 

' '  0.2272 

-6.30 

Applicant  Socioeconomic  Characteristics  (Not  Accession) 

Guem.999 

-0.0328 

0.1242 

-0.26 

Gpov.999 

-0.0696 

0.0685 

-1.02 

Ginc.999 

-0.0265 

0.0289 

-0.92 

G \et.  999 

0.0110 

0.0167 

0.66 

Grr,999 

0.0018 

0.0054 

0.33 

S.D.  of  Unobserved  Characteristics  Distn. 

Sebd 

0.0360 

0.0805 

0.45 

Sf.i 

2.7685 

0.4791 

5.78 

Sf.2 

2.0157 

0.5733 

3.52 

Sf.3 

2.7563 

0.5928 

4.65 

Sf.4 

1.4889 

0.5005 

2.97 

Sf.5 

3.8453 

2.2360 

1.72 

Sf.6 

0.8607 

0.5353 

1.61 

Sf.7 

1.1965 

0.5337 

2.24 

Sf.8 

2.1521 

0.6223 

3.46 

Sf.9 

0.2336 

0.7428 

0.31 

Sr.  iii 

0.4041 

2.1299 

0.19 

Sf.999 

8.2991 

0.8059 

10.30 
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Interpreting  the  JCM  estimation  results.  Direct  interpretation  of  the  JCM  parameter 
estimates  is  not  straightforward  given  the  interplay  among  the  various  parameters.  For  instance, 
relating  the  MOS-specific  constants  directly  to  applicant  preferences  is  not  meaningful  without 
including  the  applicant  demographic  interactions.  This  is  also  complicated  by  the  underlying 
distribution  of  unobserved  applicant  characteristics,  which  has  the  effect  of  averaging  the 
contribution  of  utility  constants  and  coefficients  to  choice  probabilities.  In  contrast,  these 
contributions  are  more  direct  under  the  MNL  model.  Given  these  practical  limitations,  we  will 
only  describe  important  observations  about  the  parameter  estimates  and  not  provide  detailed 
comparisons  of  relative  preferences  for  specific  alternatives.  We  will  then  show  how  the  JCM 
relates  higher  and  lower  priority  MOS  by  TOS  choices  for  varying  levels  of  monetary  incentives 
and  compare  the  predicted  impact  of  the  incentives  under  the  baseline  and  proposed  policies. 

We  first  describe  the  MOS  and  TOS  alternative  constants.  The  estimated  values  for 
MOS-specific  constants  range  from  about  -3  to  +3.  Note  that  these  constants  are  already 
expressed  as  differences  relative  to  MOS  1 IX  (fixed  at  0).  The  magnitudes  of  the  differences 
between  estimates  mostly  range  from  0  to  about  2.  As  shown  in  a  later  analysis,  these  differences 
are  within  the  range  of  utilities  derived  from  incentives.  Therefore  the  relative  preferences  for 
different  MOS  can  be  managed  through  selective  application  of  incentives.  Looking  at  the  three 
priority-level  1  MOS  during  this  quarter,  one  has  a  highly  negative  estimated  constant  (-4.4  for 
89D),  while  the  other  two  have  estimated  constants  (-.8  for  13F  and  -.3  for  92F)  that  are  in  the 
middle  of  the  range  of  estimated  constants.  The  high  negative  constant  for  89D  might  indicate  a 
high  average  negative  preference  given  the  nature  of  the  job  (Explosives  Ordnance  Disposal 
[EOD]  Specialist).  On  the  other  hand,  while  the  preferences  indicated  by  the  estimated  constants 
for  1 3F  and  92F  are  near  the  average,  there  might  not  be  enough  applicants  joining  these  MOS  to 
meet  Anny  requirements  -  hence  the  need  for  higher-valued  monetary  incentives  available  for 
priority  1  MOS.  Generally  speaking,  there  will  be  a  range  of  plausible  explanations  for 
interpreting  estimated  parameters.  Conversely,  interpretation  of  the  TOS-specific  estimated 
constants  is  more  straightforward.  Looking  at  these  constants  indicates  that,  in  the  absence  of 
monetary  incentives,  applicants  would  prefer  a  three  year  commitment  compared  to  a  longer 
TOS  of  four  to  six  years.  There  is  a  relatively  large  drop  in  the  estimated  constant  from  three  to 
four  years  of  TOS,  compared  to  differences  for  the  TOS  constant  from  four  to  six  years.  We 
suspect  that  this  gap  is  connected  to  the  change  in  bonus  cap  from  $10K  for  a  TOS  of  three  years 
to  $20K  for  a  TOS  of  four  to  six  years,  rather  than  an  abrupt  decrease  in  attractiveness  of  four 
years  of  TOS. 

We  next  describe  the  interaction  between  applicant  characteristics  and  enlistment 
alternatives.  We  begin  with  the  estimated  coefficients  for  the  applicant’s  AA  scores  for  different 
MOS  alternatives  and  then  describe  the  estimated  constants  corresponding  to  the  interaction 
between  applicant  demographics  and  MOS  clusters  and  TOS.  The  estimated  coefficient  for  the 
effect  of  AA  score  on  an  applicant’s  utility  for  an  MOS  is  significantly  positive.  This  indicates 
that  among  MOS  for  which  they  are  eligible,  applicants  tend  to  choose  the  MOS  for  which  they 
have  the  highest  score.  This  person-job  match  observation  is  consistent  with  previous  work 
modeling  Anny  applicants’  job  choices  (Diaz  et  ah,  2004). 

The  estimated  interaction  effects  between  applicant  demographics  and  MOS  clusters 
indicate  applicant  differences  associated  with  enlistment  in  the  Army  and  in  preferences  for 
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certain  types  of  jobs.  The  estimated  effects  for  AFQT  category  and  MOS  cluster  interaction 
indicate  that  Category  I-IIIA  applicants  are  less  likely  to  join  the  Army  than  Category  IIIB  and 
IV  applicants.  In  terms  of  types  of  MOS,  Category  I-IIIA  applicants  are  more  likely  to  enlist  in 
MOS  associated  with  Medical  and  Administrative  types  of  jobs,  and  less  likely  to  enlist  in  Close 
Combat,  Logistics/Supply,  and  Skilled  Science  Technician  (SST)  MOS.  The  estimated  effects 
for  education  status  and  MOS  cluster  interaction  indicate  that  seniors  and  applicants  with  at  least 
some  college  credits  are  less  likely  to  join  than  HS  graduates  and  non-HS  graduates.  Likely 
contributing  to  the  lower  senior  applicant’s  preference  for  joining  the  Army  is  that  school  is  still 
going  on  during  the  first  quarter,  even  taking  into  account  the  possibility  of  entering  the  Delayed 
Entry  Program  (DEP).  In  terms  of  specific  types  of  MOS,  HS  graduates  and  applicants  with  at 
least  some  college  are  more  likely  to  enlist  in  Security/Civil  Affairs  (CSA),  Administrative, 
Medical,  and  SST  clusters.  Lastly,  the  estimated  effects  for  gender  and  MOS  cluster  interaction 
indicate  that  males  are  more  likely  to  join  the  Army  than  females.  Among  jobs  with  no  gender 
restriction,  males  are  more  likely  to  enlist  in  Electronics  and  Mechanical  types  of  jobs,  while 
females  are  more  likely  to  enlist  in  Medical,  SST,  Media,  and  Logistics/Supply  types  of  jobs. 

The  estimated  interaction  effects  between  applicant  demographics  and  TOS  indicate 
some  differences  in  applicant  preferences.  The  estimated  interactions  between  AFQT  categories 
and  TOS  indicate  that  Category  I-IIIA  applicants  are  more  likely  to  enlist  for  four,  five,  and  (to  a 
lesser  degree)  six  years  than  three  years.  The  estimated  interactions  between  education  status  and 
TOS  indicate  that  HS  graduates  and  those  with  some  college  are  more  likely  to  enlist  for  four  or 
five  years  than  three  or  six  years.  Lastly,  the  estimated  interactions  between  gender  and  TOS 
indicate  that  males  are  less  likely  to  enlist  for  longer  TOS  than  females. 

We  next  discuss  results  related  to  the  monetary  incentives.  We  first  note  that  interpreting 
estimated  coefficients  for  monetary  incentives  separately  is  not  straightforward,  because  dollar 
levels  specified  by  policy  for  these  incentives  and  their  total  are  correlated.  Nevertheless,  the 
estimated  values  are  meaningful  for  inferring  the  strength  of  the  incentive’s  contribution  in 
enhancing  the  attractiveness  of  different  MOS/TOS  alternatives  to  applicants.  Overall,  the 
directions  of  the  estimated  coefficients  are  all  in  the  anticipated  direction,  with  one  exception. 
The  coefficients  for  EB,  HG,  and  AB  bonuses  and  college  money  (ACF  incentive)  indicate  that 
all  contribute  significantly  and  positively  to  applicants’  perceptions  of  utility.  The  coefficient  for 
SB  is  not  significant  (slightly  negative),  which  likely  is  due  to  correlation  among  the 
incentives.13  The  estimates  for  the  means  of  the  random  coefficients  for  reduced  EB  bonus  for 
HS  seniors  and  HS  non-seniors  are  not  significant.  While  technically  not  significant,  the 
magnitude  of  the  coefficients  is  not  negligible,  and  their  sign  is  in  the  expected  direction, 
indicating  that  seniors,  on  average,  perceive  higher  total  utility  from  the  EB/ACF  incentive 
package  than  non-seniors.14  Finally,  the  estimated  coefficient  for  the  amount  of  total  bonus  over 
the  cap  is  significantly  negative,  which  is  the  anticipated  direction.  More  importantly,  this 
negative  effect  indicates  a  potential  for  increasing  the  attractiveness  of  alternatives 


12  Only  a  single  MOS,  Chemical  Operations,  represented  the  SST  cluster  during  the  quarter. 

13  A  preliminary  analysis  using  data  from  an  earlier  quarter  in  FY  2004  with  slightly  lower  HG  bonuses  produced  a 
significant  coefficient  for  SB. 

14  This  interpretation  is  consistent  from  a  RUM  perspective,  since  seniors  are  more  likely  to  assign  greater  value  to 
the  optional  college  money  and  thus  when  choosing  between  the  EB-only  and  reduced  EB+ACF  incentive  packages 
will  elect  the  package  that  best  maximizes  utility.  In  contrast,  because  non-seniors,  on  average,  are  likely  to  perceive 
no  additional  value  from  ACF,  they  are  less  likely  to  choose  an  EB+ACF  package  with  a  greater  total  bonus  amount. 
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corresponding  to  MOS  at  the  highest  levels  (e.g.,  1  and  2)  and  TOS  of  5  or  6;  these  are  the 
alternatives  that  are  likely  to  benefit  from  raising  the  bonus  cap. 

We  next  describe  the  results  related  to  unobserved  applicant  characteristics.  The  SD  of 
unobserved  characteristics  that  interact  with  the  reduced  MOS  clusters  indicate  unequal 
unexplained  variance  in  utility  among  MOS  alternatives  (heteroscedasticity  of  random  utilities) 
and  correlation  among  alternatives  within  clusters.  Six  reduced  MOS  clusters  have  statistically 
significant  SD  estimates.  The  largest  estimated  SD  (3.8),  which  corresponds  to  the  fifth  reduced 
cluster,  has  a  /-statistic  of  1 .72  -  technically  below,  but  approaching  statistical  significance  (p  < 
.05).  The  statistically  significant  SD  estimates  and  the  differences  in  their  magnitude  indicate 
that  utilities  for  enlistment  alternatives  are  heteroscedastic  and  correlated  within  MOS  clusters. 
Lastly,  the  extra  variance  included  in  the  non-accession  utility  for  applicants  offered  more  than  a 
single  MOS  is  also  significant,  as  was  expected. 

Lastly,  we  describe  the  results  for  socioeconomic  variables  included  in  the  utility  for  non¬ 
accession.  None  among  these  variables  have  statistically  significant  estimated  coefficients,  and 
therefore  do  not  make  a  substantial  contribution  in  the  overall  applicant  decision  to  join  the 
Army.  (Note  that  this  finding  applies  to  recruits  already  going  through  processing  at  the  MEPS.) 
There  were  some  relatively  weak  effects  observed  during  preliminary  analysis  comparing 
applicants  who  actually  accessed  and  those  who  did  not  access,  but  these  effects  were  likely 
washed  out  in  the  full  JCM.  The  variable  with  the  strongest  non-significant  effect  is  the  poverty 
rate  (with  /-statistic  of  1 .02).  Its  estimated  coefficient  in  the  full  model  is  negative,  indicating  that 
applicants  in  areas  with  higher  poverty  rate  are  more  likely  to  join  the  Army. 

Bonus  Cap ’s  Effects  on  Applicant  Utilities.  Overall,  the  statistical  significance  of 
incentive-related  coefficients  indicates  that  monetary  incentives  contribute  individually  and  in 
combination  to  make  incentivized  MOS/TOS  enlistment  alternatives  more  attractive  to 
applicants,  particularly  high  quality  ones  (i.e.,  I-IIIA’s).  To  better  illustrate  the  combined  effects 
of  the  monetary  incentives,  we  plotted  high  quality  applicants’  perceived  utility  from  the 
EB/ACF,  HG,  and  SB  incentive  for  selected  MOS  priority  levels  by  TOS,  under  the  existing 
bonus  policy  (with  a  $20K  cap)  and  the  proposed  bonus  policy  (with  a  $40K  cap).15  For  this 
analysis  we  selected  two  levels  of  HG  and  SB  (in  combination),  the  first  with  both  HG  and  SB 
dollars  equal  to  $3K  and  the  second  with  HG  and  SB  dollars  equal  to  $6K.16  The  MOS  priority 
levels  selected  were  1,  2,  4,  6,  and  8.  The  EB,  reduced  EB,  and  ACF  dollar  amounts  for  these 
levels  are  shown  in  Table  5.  The  bonus  caps  for  the  baseline  policy  are  $10K  for  3  years  and 
$20K  for  4  to  6  years  TOS,  while  the  bonus  caps  for  the  proposed  policy  are  $10K  for  3  years 
and  $40K  for  4  to  6  years  TOS. 

Figure  2  shows  high  quality  applicant  estimated  perceived  utility  by  TOS  and  MOS  level 
from  EB/ACF,  HG,  and  SB  incentives  for  the  lower-valued  combination  of  HG  and  SB  under  the 
existing  bonus  cap.  Figure  3  plots  the  same  utilities  under  the  proposed  cap  of  $40K.  As 


15  We  excluded  the  AB  bonus  in  this  analysis  because:  (a)  unlike  the  HG  incentive  it  is  not  offered  to  all  incentivized 
MOS  and  (b)  the  AB  dollar  amount  ($3K)  was  relatively  low  compared  to  HG  and  SB  incentives  during  the  first 
quarter  of  FY  2005  and  subsequent  quarters.  HG  is  as  high  as  $8K  while  the  top  SB  incentive  exceeded  $10K. 

16  This  corresponds  to  HG  levels  for  30  and  60  hours  of  college  credit,  and  60-day  and  30-day  priority  levels  for  SB 
incentive. 
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evidenced  from  comparing  Figures  2  and  3,  raising  the  bonus  cap  is  expected  to  positively 
increase  the  utility  applicants  place  on  longer  TOS  and  higher  priority  MOS.  Starting  at  5  years 
of  TOS,  the  impact  of  the  increased  bonus  cap  becomes  especially  apparent,  as  the  utilities 
associated  with  priority  level  1  and  2  MOS  become  significantly  greater  than  those  associated 
with  level  4  MOS  and  below.  At  6  years  of  TOS,  an  even  greater  separation  is  achieved  for 
priority  MOS  levels  1  and  2  from  raising  the  bonus  cap.  Indeed,  under  the  existing  cap,  I-IIIA 
applicant  perceived  utility  in  choosing  a  level  1  and  2  MOS  equaled  or  was  less  than  that 
associated  with  a  level  4  MOS.  Similarly,  the  perceived  utility  associated  with  higher  priority 
MOS  for  longer  TOS  (5-6  years)  tended  to  be  equal  to  or  less  than  the  corresponding  utility 
associated  with  shorter  TOS.  In  both  cases,  these  trends  essentially  represent  an  interaction 
among  MOS  levels,  TOS,  and  total  bonus  induced  by  the  existing  bonus  cap.  Comparable 
patterns  can  be  observed  when  comparing  3-  and  4-year  TOS,  where  a  change  in  bonus  cap  from 
$10K  to  $20K  (under  baseline  policy)  or  $40K  (under  proposed  policy)  occurs;  this  effectively 
makes  it  more  attractive  for  applicants  to  enlist  for  at  least  four  years  or  more  when  considering 
higher  priority  level  MOS. 
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Utility 
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TOS 

Figure  3.  Utility  From  Incentives  Using  Proposed  Policy  (HG=$3K  and  SB=$3K) 
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Figures  4  and  5  show  the  same  information  as  Figures  2  and  3  but  using  the  higher¬ 
valued  combination  of  HG  and  SB.  There  are  two  important  observations  immediately  apparent 
when  examining  these  two  additional  figures.  First,  immediately  apparent  from  Figures  3  and  4 
is  the  higher  overall  levels  of  the  utilities,  which  clearly  is  due  to  the  higher  HG  and  SB  bonuses. 
Second,  as  evident  by  comparing  Figures  4  and  5,  the  low  bonus  caps  under  the  baseline  policy 
have  a  highly  constraining  effect  under  higher  levels  of  HG  and  SB  incentives,  resulting  in 
diminished  discriminatory  potential  for  the  various  MOS  priority  levels.  Enlistment  alternatives 
with  4  years  of  TOS  or  higher  clearly  benefited  from  higher  bonus  caps  in  the  proposed  policy. 
Clearly  this  is  because  the  bonus  cap  for  3  years  of  TOS  remains  the  same  at  $10K  under  the 
proposed  policy,  which  is  easily  exceeded  with  higher  levels  of  HG  and  SB.  (The  top  SB 
incentive  amount  exceeding  $  1  OK,  which  became  available  subsequent  to  our  analysis  sample 
quarter,  in  most  likelihood  would  have  induced  eligible  applicants  to  sign-up  for  at  least  four 
years  under  the  new  bonus  cap.)  In  addition  to  making  4  years  TOS  or  higher  more  attractive 
than  3  years,  the  higher  bonus  cap  under  the  proposed  policy  also  fixed  an  undesirable  effect  of 
the  interaction  induced  by  the  bonus  cap  on  the  relative  utility  levels  of  higher  level  MOS.  For 
example,  in  Figure  4  the  order  of  the  utility  values  for  MOS  levels  1 , 2,  and  4  is  opposite  their 
intended  effect,  but  is  corrected  in  Figure  5  under  a  higher  bonus  cap.  This  undesirable 
interaction  induced  by  the  bonus  cap  on  longer  TOS  and  higher  level  MOS  is  again  expected  to 
become  an  issue  as  soon  as  the  HG  and  SB  incentives  (or  other  incentives  not  based  on  MOS 
levels)  reach  a  certain  level.17 


17  For  this  reason,  the  SB  incentive  structure  in  the  proposed  policy,  which  closely  tied  the  SB  dollar  amounts  to  the 
MOS  levels,  is  a  favorable  approach.  It  stands  a  better  chance  of  preventing  the  undesirable  interaction  effect  at 
higher  MOS  level  and  longer  TOS,  and  is  therefore  more  likely  to  produce  enlistments  that  are  consistent  with  MOS 
priority  levels. 
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Figure  4.  Utility  From  Incentives  Using  Baseline  Policy  (HG-$6K  and  SB=$6K) 
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Figure  5.  Utility  From  Incentives  Using  Proposed  Policy  (HG=$6K  and  SB=$6K) 
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Model  Fit  Diagnostics.  While  its  interpretation  is  not  straightforward,  the  overall  fit  of 
the  model  (pseudo-R2  =  .24)  is  substantial  given  the  large  dimension  of  the  choice  space. 
Additionally,  this  level  of  fit  compares  favorably  to  that  typically  observed  in  the  applied  social 
and  behavioral  sciences.  To  further  evaluate  model  fit,  we  compared  the  expected  choices  of 
applicants  based  on  the  estimated  JCM  to  their  actual  MOS  and  TOS  choices.  This  comparison 
was  conducted  separately  using  the  JCM  estimation  sample  («  =  4,020)  and  the  hold-out 
validation  sample  (n  =  14,783).  Within  each  sample,  comparisons  between  expected  and 
observed  choices  were  carried  out  for  the  overall  sample  and  by  subgroups. 

Results  of  these  analyses  for  the  estimation  sample  can  be  found  in  Tables  A-l  and  A-2 
in  Appendix  A.  Each  row  in  these  tables  compares  the  observed  and  expected  number  of 
accessions  and  the  corresponding  percentage  within  a  subgroup  (e.g.,  sum  of  percentages  equals 
100  within  male  subgroup),  for  each  MOS  and  TOS  enlistment  alternative.  The  column  “Diff. 
N.”  reports  the  difference  between  observed  and  expected  number  of  accessions,  while  the 
column  “Ratio  N.”  reports  the  ratio  of  expected  accessions  relative  to  observed  accessions.  As 
evidenced  by  Tables  A-l  and  A-2,  the  estimated  number  of  accessions/non-accessions  closely 
matched  the  observed  accessions/non-accessions  for  most  MOS  alternatives  and  all  TOS 
alternatives,  both  for  the  sample  as  a  whole  and  by  subgroup.  While  there  were  a  few  MOS 
alternatives  for  which  there  were  somewhat  sizeable  differences,  they  tended  to  be  for  MOS  and 
subgroup  combinations  where  the  number  of  accessions  was  very  small. 

Results  for  the  hold-out  sample  can  be  found  in  Tables  A-3  and  A-4  in  Appendix  A.  As 
with  the  estimation  sample,  there  was  a  strong  correspondence  between  the  observed  and 
expected  number  of  accessions/non-accessions  for  most  MOS  alternatives  and  all  TOS 
alternatives,  both  for  the  overall  sample  and  by  subgroup.  Similarly,  where  there  were  relatively 
large  differences,  they  tended  to  occur  for  combinations  of  MOS  and  subgroups  with  small 
accessions. 

Overall  our  fit  diagnostics  indicated  that  the  internal  fit  and  predictive  accuracy  of  the 
JCM  were  very  good,  both  for  the  sample  as  a  whole  and  by  subgroup. 

Summary.  The  estimated  JCM  showed  differences  in  preferences  among  MOS 
alternatives  and  years  of  TOS,  holding  applicant  characteristics  and  incentives  constant.  The 
estimated  JCM  also  indicated  that  there  are  applicant  differences  in  MOS/TOS  preferences. 
Lastly,  the  results  related  to  monetary  incentives  demonstrates  potential  for  classification  that  is 
consistent  with  enlistment  priorities  of  the  Army.  In  particular,  choice  probabilities  for  “higher 
level”  MOS  and  longer  TOS  by  targeted  quality  applicants  (e.g.,  those  in  AFQT  category  I-IIIA 
and  with  30  hours  of  college  credit  or  higher)  are  expected  to  increase  under  the  higher  bonus 
caps  of  the  proposed  incentive  policy.  Findings  from  our  simulations  presented  later  lend  support 
to  these  observations. 


18  In  aggregating  accessions  by  MOS/TOS  choice,  the  number  applicants  was  weighted  to  reflect  their  representation 
in  the  total  analysis  sample  (i.e.,  the  estimation  and  hold-out  samples  combined). 


25 


Forecasting  Impact  of  Changes  in  Bonus  Policy 


To  investigate  the  impact  of  raising  the  bonus  cap  on  Army  accessions,  we  compared  the 
estimated  MOS/TOS  classification  distribution  of  applicants  using  the  full  analysis  sample  ( n  = 
18,803)  under  two  bonus  structures: 

1 .  the  existing  bonus  structure,  in  place  during  the  period  for  which  the  data  were  collected, 
where  the  bonus  cap  is  $20K;  and 

2.  a  modified  bonus  structure,  based  on  proposed  incentive  levels  made  at  the  Enlistment 
Incentive  Review  Board  (EIRB)  meeting  on  16  February  2006,  where  the  bonus  cap  was 
raised  to  $40K. 

Comparisons  were  made  under  current  market  conditions  during  the  first  quarter  of  FY 
2005  (i.e.,  using  the  full  applicant  sample)  and  under  illustrative  market  expansion  rates,  where 
raising  the  bonus  cap  to  $40K  was  assumed  to  expand  the  Army’s  high  quality  applicant  pool 
(see  Appendix  F).  The  following  sections  describe  the  experimental  conditions  used  in  the 
constructive  simulations  and  the  evaluation  indices  used  to  analyze  the  forecasted  impact  of 
raising  the  bonus  cap  on  Army  accessions. 

Design  and  Conditions 

Several  independent  simulations  were  conducted,  in  each  case  using  all  applicants  in  the 
analysis  sample.  The  first  two  focus  on  channeling  effects  under  assumed  zero  market  expansion, 
and  the  last  three  introduce  market  expansion.  Each  simulation  produced  a  simulated  MOS-TOS 
classification  distribution  based  on  the  aggregated  applicant  enlistment  choice  probabilities 
estimated  using  the  JCM.  For  the  first  (i.e.,  baseline)  simulation,  the  JCM  choice  probabilities 
were  computed  using  the  existing  bonus  structure  in  place  during  the  first  quarter  of  FY  2005. 

For  the  remaining  simulations,  JCM  probabilities  were  computed  using  the  bonus  structure 
proposed  during  thel6  February  2006  EIRB  meeting.  Detailed  comparisons  of  the  two  bonus 
structures  are  shown  in  Table  4. 

For  analysis  purposes,  the  estimated  MOS/TOS  classification  distribution  obtained  from 
the  first  simulation  served  as  the  baseline  distribution.  Each  of  the  MOS/TOS  classification 
distributions  from  the  other  four  simulations  were  then  compared  against  this  baseline. 
Comparing  these  subsequent  simulations  against  the  baseline  distribution,  as  opposed  to  the 
observed  MOS/TOS  distribution,  is  preferable  since  differences  in  choice  pattern  can  be  more 
meaningfully  attributed  to  differences  in  bonus  structure  and/or  market  conditions. 

As  shown  in  Table  4,  where  the  two  bonus  structures  differ  is  in  the  EB  bonus  amounts 
for  priority  level  1  MOS.  Otherwise,  all  other  bonuses  are  the  same  by  TOS  across  the  two 
structures.  While  the  higher  cash  bonus  by  itself  was  expected  to  increase  the  attractiveness  of 
level  1  MOS,  the  raised  bonus  cap’s  greatest  impact  was  expected  to  be  in  channeling  applicants 
to  longer  TOS.  This  can  be  seen  in  Table  5,  which  shows  the  average  pre-cap  bonus  amounts 
over  the  existing  cap  of  $20K  for  select  HG,  SB,  and  EB/ACF  levels.  As  suggested  by  the  table, 
raising  the  bonus  cap  to  $40K  would  increase  the  average  bonus  awarded  significantly.  This  is 
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especially  true  for  higher  quality  applicants  (I-IIIA’s)  signing  up  for  higher  priority  MOS  (levels 
1  and  2)  and  longer  TOS.  Thus,  even  under  FY  2005  HG  and  SB  bonus  levels,  raising  the  bonus 
cap  to  $40K  can  be  expected  to  increase  the  attractiveness  of  high  priority  MOS  and  longer  TOS 
for  targeted  applicants.  The  simulation  results  discussed  in  the  next  section  provide  support  for 
these  expectations. 

To  isolate  the  effects  of  raising  the  bonus  cap,  we  kept  the  SB  dollar  amount  constant 
across  the  two  bonus  structures.  We  decided  against  using  the  proposed  SB  levels  because  the 
dollar  amounts  were  almost  twice  those  offered  in  the  first  quarter  of  FY  2005.  Thus,  using  the 
proposed  SB  incentives  would  have  resulted  in  bonuses  well  in  excess  of  the  $20K  bonus  cap 
under  the  baseline  condition.  Additionally,  there  were  significant  differences  in  how  the 
proposed  SB  levels  were  to  be  applied  compared  to  the  first  quarter  of  FY  2005.  Specifically, 
while  some  information  indicated  that  higher  dollar  amounts  were  to  go  to  top  priority  seats  by 
MOS  level  ($15K  for  Levels  1  and  2  and  $10K  for  level  3),  the  MILPER  message  for  the  first 
quarter  of  FY  2005  simply  stated  “as  needed.”  Preliminary  baseline  simulation  runs  using  the 
proposed  SB  dollar  amounts  and  structure  confirmed  that  they  were  problematic.  From  these 
runs,  we  obtained  percentage  shares  for  priority  levels  1  and  2  MOS  that  were  substantially 
lower  than  their  actual  (or  fitted)  percentages,  indicating  that  the  proposed  SB  amounts  and 
structure  were  not  consistent  with  the  actual  enlistment  behavior  of  applicants  in  the  analysis 
sample;  specifically,  that  level  1  and  2  MOS  were  less  attractive  than  indicated  by  applicants’ 
actual  enlistment  choices. 
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Table  4.  Bonus  Dollar  Amounts  by  Priority  Level  and  Type  Under  Baseline  and  Proposed  Policy  Condition 
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The  full  EB  dollar  amounts  (i.e.,  EB$  Only)  by  MOS  level  are  based  on  the  proposed  incentive  policy  structure  during  the  16  February 
2006  EIRB  meeting.  The  reduced  EB  amounts  (i.e.,  in  EBS+ACF)  are  based  on  approximated  values. 

The  dollar  amounts  for  the  HG  and  SB  bonus  under  the  Proposed  Policy  condition  were  fixed  to  corresponding  levels  under  the  Baseline 


Table  5.  Bonus  Dollar  ($l,OOOs)  Amounts  Over  the  $20K  Cap  (TOS-4,5,6)  or  $10K  Cap 
(T0S=3)  for  Selected  Levels  ofHG,  SB,  and  EB/ACF Incentive  Level 


Incentive  Level 

TOS 

HG  SB 

EB/ACF 

3 

4 

5 

6 

BA  ($8K)  30-day  ($9K) 

1 

14 

9 

13 

17 

2 

13 

7 

11 

13 

3 

12 

5 

9 

11 

4 

11 

3 

5 

9 

5 

10 

1 

3 

5 

6 

9 

1 

3 

7 

8 

1 

30-day  ($6K) 

1 

11 

6 

10 

14 

2 

10 

4 

8 

10 

3 

9 

2 

6 

8 

4 

8 

2 

6 

5 

7 

2 

60-day  ($3  K) 

1 

8 

3 

7 

11 

2 

7 

1 

5 

7 

3 

6 

3 

5 

4 

5 

3 

N.A. 

1 

5 

4 

8 

2 

4 

2 

4 

3 

3 

2 

60-Hrs  ($6K)  30-day  ($9K) 

1 

12 

7 

11 

15 

2 

11 

5 

9 

11 

3 

10 

3 

7 

9 

4 

9 

1 

3 

7 

5 

8 

1 

3 

6 

7 

1 

30-day  ($6K) 

1 

9 

4 

8 

12 

2 

8 

2 

6 

8 

3 

7 

4 

6 

4 

6 

4 

60-day  ($3K) 

1 

6 

1 

5 

9 

2 

5 

3 

5 

3 

4 

1 

3 

4 

3 

1 

N.A. 

1 

3 

2 

6 

2 

2 

2 

30-Hrs  ($3K)  30-day  ($9K) 

1 

9 

4 

8 

12 

2 

8 

2 

6 

8 

3 

7 

4 

6 

4 

6 

4 

30-day  (S6K) 

1 

6 

1 

5 

9 

2 

5 

3 

5 

3 

4 

1 

3 

4 

3 

1 

60-day  (S3K) 

1 

3 

2 

6 

2 

2 

2 

N.A. 

1 

3 

Note.  HG,  SB,  and  EB  combinations  not  exceeding  bonus  cap  of  $20K  are  excluded. 


29 


Analysis  Indices 

We  used  three  types  of  analysis  indices  to  evaluate  the  raised  bonus  cap’s  impact.  Each 
index  summarizes  comparisons  between  the  baseline  and  each  one  of  the  five  policy  simulation 
conditions.  Two  indices  measure  channeling  effects  by  MOS  and  TOS,  while  the  third  index 
measures  the  increase  in  bonuses  awarded.  These  indices  were  computed  for  the  overall  sample 
and  by  subgroup  using  AFQT  categories,  education  status,  and  gender.  The  discussions  below 
provide  a  conceptual  and  computational  description  of  each  index. 

Channeling  Effects  by  MOS.  The  first  type  of  index  measures  the  raised  bonus  cap’s 
potential  channeling  effects  on  applicants’  MOS  choice.  For  each  MOS,  it  expresses  the 
expected  increase/decrease  in  accessions  under  the  proposed  bonus  structure,  for  given  market 
expansion  rate,  as  a  percentage  of  expected  baseline  accessions.  This  can  be  expressed 
computationally  as: 


N,(m) 

where  N p(m,r)  is  the  expected  accessions  for  the  mth  MOS  under  the  proposed  bonus  structure 
and  a  market  expansion  rate  of  r,  and  NB(m )  is  expected  accessions  for  the  /nth  MOS  under  the 
baseline  condition.  Expected  accessions  were  estimated  using  the  aggregated  JCM  choice 
probabilities  of  all  applicants  for  the  /nth  MOS  under  each  bonus  structure  condition: 

m,t  |  Baseline) 

Np(m,r)-  J>(/|  r)P(i,m,t  |  Policy;  r) 

For  each  summation  above,  the  JCM  choice  probabilities  of  applicants  were  aggregated  across 
all  TOS.  The  second  summation  is  a  weighted  aggregation  of  choice  probabilities  under  the 
policy  case.  For  instance,  under  the  3%  market  expansion  condition,  W(i  |  r)  equals  1 .03  if  the 
zth  applicant  is  I-IIIA  and  unity  otherwise.19 

To  measure  the  potential  channeling  effects  from  lower  to  higher  priority  MOS  by 
subgroup,  we  computed  the  index  following  the  same  procedure,  but  using  only  applicants 
belonging  to  the  targeted  subgroup: 


CMc(m ) 


N  p  r(m,r)-  N  „  Am) 

P'cy  ’  .B'cy  xlOO 


The  subscript  G  identifies  one  of  the  following  eight  subgroups: 


19  In  interpreting  the  computations  used  for  these  indices,  a  useful  heuristic  is  to  view  the  applicant  choice 
probabilities  produced  by  the  JCM  as  being  attributed  to  partial  individuals. 
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•  AFQT  Category: 

(1)  I-IIIA;  (2)  IIIB  and  Lower 

•  Education  Status: 

(3)  Some  College;  (4)  H.S.  Graduate;  (5)  Senior;  (6)  Not  H.S.  Graduate 

•  Gender: 

(7)  Male;  (8)  Female 

The  subgroup  version  of  the  expected  accessions  Npc(m,r )  and  NBG(m)  was  computed  in  the 

same  way  as  in  the  overall  index,  but  the  choice  probabilities  were  aggregated  exclusively  using 
those  applicants  belonging  to  the  targeted  subgroup  G. 


Channeling  Effect  by  TOS.  The  second  type  of  index  measures  the  degree  to  which  the 
raised  bonus  cap  channel  applicants  from  shorter  to  longer  TOS.  Similar  to  the  preceding  index 
for  MOS,  this  index  represents  the  relative  increase/decrease  in  expected  accessions  signing  up 
for  a  select  TOS  under  the  proposed  bonus  structure,  for  a  given  market  expansion  rate,  as  a 
percentage  of  the  expected  baseline  accessions.  The  index  is  computed  for  the  overall  sample 
and  by  subgroup,  respectively,  using: 


CT(l)  =  tAhli  X  l  oo 

Nb(<) 

t\  N  pr{t,r)-N  Rr(t) 
CTG(t)  =  P’G  , .  g>  xlOO 


where  Np(t,r)  is  the  number  of  expected  accessions  signing-up  for  t  years  of  TOS  under  the 
proposed  bonus  structure  and  a  market  expansion  rate  of  r,  and  NB(t)  is  the  number  of  expected 
accessions  signing-up  for  t  years  of  TOS  under  the  baseline  condition.  The  subgroup  quantities 
N P,G  (b  r)  and  A'B  G  (/)  are  the  same  as  their  respective  counterparts  in  the  overall  index,  except 
that  expected  accessions  were  counted  only  for  the  applicable  subgroup  G. 

Increases  in  Bonus  Award  Amounts.  The  third  type  of  index  measures  the  raised  bonus 
cap’s  impact  on  the  average  bonus  dollar  amount  awarded  to  applicants.  For  each  MOS,  the 
index  represents  the  difference  in  average  total  bonus  per  applicant  under  proposed  and  baseline 
bonus  structure.  The  index  is  computed  for  the  overall  sample  and  by  subgroup,  respectively, 
using: 


DB(m )  =  Bp(m)~  BB(m) 

DBg  (m)  =  Bpc  {m)  -  Bb  g  {m) 

where  Bp(m)  and  BPG{m)  are  the  average  bonuses,  respectively,  for  overall  and  subgroup  G 
under  the  proposed  bonus  structure,  and  BB(m )  and  BBG(m)  are  corresponding  averages  under 

the  baseline  bonus  structure.  The  average  bonuses  for  the  overall  sample  were  computed  using 
the  following  procedures: 
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BB{m)  = 


YM  ,m,t  |  Baseline).P(z,m,£  |  Baseline) 

i,( _ 

|  Baseline) 

i,( 

|  Policy)p(z,m,/ 1  Policy;  r-0) 

i,t _ 

]!T P(i,m,t  |  Policy;  r  =  0) 

ij 


As  in  the  computation  of  the  channeling  effects  indices,  it  is  again  useful  to  view  the  choice 
probabilities  above  as  attributed  to  partial  individuals.  The  quantities  TB(i,m,t  |  Baseline)  and 
TB(i  ,m,t  |  Policy)  are  the  combined  total  of  all  bonuses  (i.e.,  EB,  HG,  SB,  and  AB)  that  the  z'th 
applicant  would  receive  if  he/she  signs  up  for  the  mth  MOS  for  t  years  of  TOS  under  the  baseline 
and  policy  incentive  structures,  subject  to  the  applicable  bonus  cap.  For  example,  an  applicant 
with  pre-cap  bonus  total  of  $25K  would  only  receive  $20K  total  under  the  baseline  for  a  4-year 
TOS  and  all  of  the  $25K  under  the  proposed  bonus  structure.  The  subgroup  average  bonuses  are 
computed  as  above,  but  the  summation  is  evaluated  only  across  applicants  belonging  to  the 
targeted  subgroup. 

As  can  be  seen,  differences  in  average  total  bonuses  can  be  positive  or  negative.  This 
index  is  a  function  of  both  increases  in  bonuses  and  channeling  effects  under  the  proposed 
incentive.  The  increases  in  total  bonuses  under  the  proposed  bonus  policy  are  expected  to 
produce  higher  average  bonuses  across  all  MOS,  with  relatively  higher  increments  anticipated 
for  higher  MOS  levels.  However,  the  channeling  of  quality  applicants  who  qualify  for  higher 
bonuses  away  from  some  MOS  (especially,  lower  level  MOS)  is  expected  to  lower  their  average 
total  bonus. 
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Findings 


Forecasted  Impact  on  Applicant  Job  Choice 

Figures  6  through  10  illustrate  the  forecasted  impact  of  raising  the  bonus  cap  to  $40K  on 
applicants’  job  choice  by  MOS  and  select  subgroups  under  zero  market  expansion.20  Overall, 
and  as  these  figures  show,  raising  the  cap  is  expected  to: 

■  Increase  somewhat  the  percentage  of  Army  applicants  accessing. 

■  Uniformly  channel  applicants,  particularly  higher  quality  applicants  (i.e.,  I-IIIA’s  or  those 
with  some  college),  to  higher  priority  MOS  and  away  from  less  incentivized  ones. 

More  specifically: 

■  Across  all  applicants,  the  percent  of  those  not  accessing  is  expected  to  decline  about 
0.4%.  Among  higher  quality  applicants,  quality  taken  as  I-IIIA’s  or  HSDG  or  greater, 
non-accessions  are  projected  to  decrease,  on  average,  0.5-1%. 

■  Across  all  applicants,  accessions  to  higher  priority  MOS  are  projected  to  increase,  on 
average,  about  6.6%,  ranging  from  under  1%  (1 IX,  18X)  to  21%  (89D)  (see  Figure  6). 
Whereas,  accessions  to  lower  priority  or  less  incentivized  MOS  are  expected  to  decrease, 
on  average,  about  1.4%.  Among  the  lower  priority  MOS,  19D,  19K,  LEI,  and  EL2 
would  be  the  most  markedly  affected  (percent  decreases  greater  than  2%). 

■  Among  higher  quality  applicants,  accessions  to  higher  priority  MOS  are  expected  to 
increase,  on  average,  about  8-10%,  while  accessions  to  lower  priority  MOS  are  expected 
to  decline  roughly  2%  (see  Figures  7-9).  According  to  our  forecast,  this  channeling  effect 
will  be  most  pronounced  among  those  with  some  college  experience,  one  of  the  primary 
targets  of  this  policy  change.  Among  this  group,  accessions  to  higher  priority  MOS 
would  increase,  on  average,  about  21%.  Conversely,  accessions  to  lower  priority  MOS 
would  decline  about  4%. 

■  While  comparable,  there  are  some  aggregate  differences  in  forecasted  accession  patterns 
for  males  and  females  (see  Figure  10).  For  example,  among  males,  accessions  to  higher 
priority  MOS  are  expected  to  increase  about  6.5%,  while  among  females,  accessions  are 
expected  to  increase  about  10.9%.  Nevertheless,  because  the  numbers  behind  these 
percentages  are  comparatively  small,  there  is  very  little  indication  that  they  would 
translate  into  significant  changes  in  the  composition  of  those  accessing,  either  to  the 
Army  as  a  whole  or  to  specific  MOS. 


20  For  a  complete  breakdown  of  results,  see  Appendix  B. 
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Figure  7.  Percent  Change  in  Accessions  Relative  to  Baseline  by  MOS  and  AFQT  Category 

Under  Zero  Market  Expansion 
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Figure  8.  Percent  Change  in  Accessions  Relative  to  Baseline  by  MOS  and  Educational 
Attainment  Under  Zero  Market  Expansion 
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Figure  9.  Percent  Change  in  Accessions  Relative  to  Baseline  by  MOS  and  Educational 
Attainment  (Some  College  and  High  School  Graduates  Only)  Under  Zero  Market  Expansion 
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Forecasted  Impact  on  Applicant  TOS  Choice 


Figures  1 1  through  1 5  show  the  forecasted  impact  of  raising  the  bonus  cap  to  $40K  on 
applicants’  term-of-service  (TOS)  choice  by  MOS  priority  level  and  select  subgroups  under  zero 
market  expansion.21  Consistent  with  the  preceding  analyses,  and  as  these  figures  demonstrate, 
raising  the  cap  is  expected  to  channel  applicants,  particularly  higher  quality  applicants  (i.e.,  I- 
IIIA’s  or  those  with  some  college),  to  somewhat  longer  TOS  for  higher  priority  MOS. 
Specifically: 

■  Across  all  applicants,  TOS  commitments  among  higher  priority  MOS  are  projected  to 
increase,  on  average,  about  4.0%  for  4  years,  1 1.0%  for  5  years,  and  13.6%  for  6  years 
(see  Figure  11),  while  declining  1 .3%  for  3  years.  Practically,  this  translates  into  a 
modest  increase  in  the  average  TOS  from  3.82  years  to  3.86  years. 

■  Among  higher  quality  applicants,  TOS  commitments  to  higher  priority  MOS  are  expected 
to  increase,  on  average,  roughly  3.5-5%  for  4  years,  12-16%  for  5  years,  and  14-17%  for 
6  years,  while  declining  1.5-3. 5%  for  3  years  (see  Figures  12-14) — an  increase  in  the 
average  TOS  from  4.10  years  to  4.15  years.  As  with  the  preceding  analyses,  this  trend 
will  be  most  pronounced  among  applicants  with  some  college  experience,  where  TOS 
commitments  to  higher  priority  MOS  are  projected  to  increase  13%  for  4  years,  about 
21%  for  5  years,  and  19%  for  6  years.  Among  applicants  with  some  college  experience, 
this  represents  an  increase  in  their  average  TOS  from  4.25  to  4.32  years. 

■  Among  males  and  females,  TOS  commitments  to  higher  priority  MOS  are  projected  to 
increase,  although  at  a  comparatively  higher  rate  for  females.  As  Figure  1 5  shows, 
among  males,  TOS  commitments  to  higher  priority  MOS  are  expected  to  increase  about 
4%  for  4  years,  11%  for  5  years,  and  13%  for  6  years — an  increase  in  average  TOS  from 
3.81  to  3.85  years.  Among  females,  TOS  commitments  to  higher  priority  MOS  are 
expected  to  increase  about  7%  for  4  years,  1 7%  for  5  years,  and  1 9%  for  6  years — an 
increase  in  average  TOS  from  3.93  to  3.99  years. 


21  For  a  complete  breakdown  of  results,  see  Appendix  C. 
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TOS 

Note.  A=A11  Levels;  H=Levels  1-2,  L=Levels  3-9. 
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Figure  14.  Percent  Change  in  Accessions  Relative  to  Baseline  by  Term  of  Service  (TOS), 
Incentive  Level,  and  Educational  Attainment  (Some  College  and  High  School  Graduates  Only) 

Under  Zero  Market  Expansion 


Figure  15.  Percent  Change  in  Accessions  Relative  to  Baseline  by  Term  of  Service  (TOS) 
Incentive  Level,  and  Gender  Under  Zero  Market  Expansion 


Forecasted  Impact  on  Bonus  Dollars  Awarded 


Figures  16  through  20  show  the  forecasted  impact  of  raising  the  bonus  cap  to  $40K  on  average 
bonus  dollars  (i.e.,  sum  of  EB,  FIG,  SB,  and  AB)  awarded  by  MOS  and  select  subgroups  under 
zero  market  expansion.22  Overall,  raising  the  cap  is  expected  to  increase  the  average  bonus 
dollars  awarded  from  $5,633  to  $6,092 — an  increase  of  8%.  More  specifically,  our  analyses 
indicate  that: 

■  Across  all  applicants,  the  average  bonus  dollars  awarded  to  applicants  accessing  to  higher 
priority  MOS  are  projected  to  increase,  on  average,  by  $1,267  (see  Figure  16)  —  from 
$9,530  to  $10,797  per  accession.  Taken  together,  this  translates  into  an  increase  in  the 
total  bonus  dollars  awarded  by  Army  from  $77.5  M  to  $83.9  M. 

■  Among  higher  quality  applicants,  the  average  bonus  dollars  awarded  to  those  accessing  to 
higher  priority  MOS  is  expected  to  increase,  on  average,  by  $  1 ,700-$3,300  —  a  1 3-23% 
increase  (see  Figures  17-19).  Of  the  high  priority  MOS,  13F,  89D,  and  92F  would 
experience  the  greatest  increases  —  about  $3,000-$7,000  each.  Among  applicants  with 
some  college  experience,  the  average  bonus  dollars  awarded  to  those  accessing  to  higher 
priority  MOS  is  forecasted  to  increase,  on  average,  by  $3,285  (or  23%). 

■  Among  males  and  females,  the  average  bonus  dollars  awarded  to  those  accessing  to 
higher  priority  MOS  are  expected  to  increase  about  $1,200  and  $2,100,  respectively.  As 
Figure  20  indicates,  among  males,  average  bonus  dollars  awarded  would  increase  by 
$1,21 1,  from  $9,674  to  $10,885  —  a  12%  increase.  Among  females,  the  average  bonus 
awarded  would  increase  by  $2,081,  from  $7,548  to  $9,629  -  a  28%  increase. 


22  For  a  complete  breakdown  of  results,  see  Appendices  D  and  E. 
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Figure  16.  Change  in  Average  Bonus  Awarded  (in  Thousands  of  Dollars)  Relative  to  Baseline  by 

MOS  Under  Zero  Market  Expansion 
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Figure  1 7.  Change  in  Average  Bonus  Awarded  (in  Thousands  of  Dollars)  Relative  to  Baseline  by 
MOS  and  AFQT  Category  Under  Zero  Market  Expansion 
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Figure  18.  Change  in  Average  Bonus  Awarded  (in  Thousands  of  Dollars)  Relative  to  Baseline  by 
MOS  and  Educational  Attainment  Under  Zero  Market  Expansion 
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Figure  19.  Change  in  Average  Bonus  Awarded  (in  Thousands  of  Dollars)  Relative  to  Baseline  by 
MOS  and  Educational  Attainment  (Some  College  and  High  School  Graduates  Only)  Under  Zero 

Market  Expansion 
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Figure  20.  Change  in  Average  Bonus  Awarded  (in  Thousands  of  Dollars)  Relative  to  Baseline  by 

MOS  and  Gender  Under  Zero  Market  Expansion 
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Discussion 


Recap  of  Key  Findings 

Taken  together,  our  analyses  indicated  the  following: 

■  Raising  the  current  bonus  cap  to  $40K  is  expected  to  increase  somewhat  overall  Army 
accessions  and  to  uniformly  channel  applicants,  particularly  high  quality  applicants  (i.e., 
I-IIIA ’s  or  those  with  some  college),  to  higher  priority  MOS  and  away  from  lower 
priority  ones.  Specifically,  raising  the  cap  is  projected  to  decrease  the  percent  of  non¬ 
accessions  by  0.4%,  while  accessions  to  higher  priority  MOS  are  expected  to  increase,  on 
average,  6.6%.  Among  higher  quality  applicants,  non-accessions  are  expected  to  decline 
by  0.5-1%.  Similarly,  accessions  to  higher  priority  MOS  are  projected  to  increase,  on 
average,  about  8-10%,  while  accessions  to  lower  priority  MOS  are  expected  to  decrease 
by  2%. 

■  Consistent  with  its  impact  on  accessions,  raising  the  cap  is  expected  to  attract  applicants, 
particularly  higher  quality  applicants,  to  somewhat  longer  TOS  for  higher  priority  MOS. 
Specifically,  TOS  commitments  among  higher  priority  MOS  are  projected  to  increase,  on 
average,  about  4.0%  for  4  years,  1 1 .0%  for  5  years,  and  13.6%  for  6  years,  while 
declining  1.3%  for  3  years.  Similarly,  among  higher  quality  applicants,  TOS 
commitments  are  expected  to  increase,  on  average,  roughly  3.5-5%  for  4  years,  12-16% 
for  5  years,  and  14-17%  for  6  years,  while  declining  1.5-3. 5%  for  3  years — an  increase  in 
the  average  TOS  from  4.10  years  to  4.15  years. 

■  On  average,  raising  the  cap  is  expected  to  increase  the  bonus  dollars  awarded  per 
accession  from  to  $5,633  to  $6,092 — an  increase  of  8%.  Across  all  applicants,  the 
average  bonus  dollars  awarded  to  applicants  accessing  to  higher  priority  MOS  are 
projected  to  increase,  on  average,  by  $1 ,267.  Overall,  this  translates  into  an  increase  in 
the  total  bonus  dollars  awarded  by  Army  from  $77.5  M  to  $83.9  M.  Among  higher 
quality  applicants,  the  average  bonus  dollars  awarded  to  those  accessing  to  higher  priority 
MOS  is  expected  to  increase,  on  average,  by  $l,700-$3,300  —  a  13-23%  increase. 


Recommendations 

We  used  the  mixed  multinomial  logit  model  to  describe  MOS-TOS  training  choices  of 
Army  applicants  during  the  first  quarter  of  FY  2005  as  a  function  of  applicant  characteristics, 
incentives,  and  extant  bonus  cap  policy.  The  estimated  model  was  subsequently  employed  to 
predict  training  choices  under  a  modified  incentive  policy  with  a  higher  bonus  cap.  While  the 
internal  predictive  accuracy  of  the  model  was  validated  using  a  separate  hold-out  sample  from 
the  same  quarter  under  extant  policy  and  zero  market  expansion,  the  predicted  accuracy  under 
the  new  incentive  policy  and  increased  bonus  cap  was  not  validated  because  the  needed  data 
were  not  available  at  the  time  of  the  study. 
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The  choice  model  and  policy  simulation  capabilities  developed  in  this  study  have  the 
potential  to  inform  the  EIRB  and  other  Army  staff  in  charge  of  managing  the  Army’s  enlistment 
incentives.  For  instance,  the  estimated  choice  model  could  be  embedded  in  a  computer 
application  for  simulating  incentive  policy  scenarios.  As  preparation  for  such  application,  the 
estimated  model  should  be  validated  under  the  new  incentive  policy  and  increased  bonus  cap. 

We  outline  two  approaches  for  conducting  this  validation.  In  the  first  approach,  the  procedure 
used  to  validate  the  internal  predictive  accuracy  would  be  applied  directly  to  applicant 
transactions  data  obtained  under  the  new  incentive  policy  and  bonus  cap,  and  predicted  and 
observed  choices  compared  by  subgroups.  This  approach  would  likely  require  additional 
calibration  of  the  estimated  MOS  and/or  TOS  constants  in  the  model  to  reflect  differences  in 
demand  during  the  first  quarter  of  FY  2005  and  the  new  period  from  which  new  data  would  be 
collected.  The  disadvantage  of  this  approach  is  that  it  requires  individual  transactions  data,  which 
are  difficult  to  obtain  and  to  process.  In  the  second  approach,  the  observed  choices  from  the  new 
period  would  be  compared  to  the  predicted  choices  from  the  first  quarter  of  FY  2005  by 
subgroup.  In  addition  to  the  calibration  of  MOS  and  TOS  constants  to  account  for  changes  in 
demands  between  the  two  periods,  this  approach  would  include  re-weighting  the  first  quarter  of 
FY  2005  data  to  reflect  the  applicant  subgroups’  distribution  in  the  new  data.  This  approach 
overcomes  the  difficulty  of  collecting  and  processing  individual  transaction  data,  because  it  only 
requires  applicant  demographics  and  test  scores  and  their  MOS-TOS  reservations. 

The  JCM  developed  in  this  study  empirically  describes  individual  characteristics  (test 
scores  and  demographics)  and  job  preferences  under  the  existing  classification  system.  It  is 
therefore  a  logical  starting  point  for  personnel  classification  interventions  studies.  For  example, 
using  a  framework  with  a  JCM  as  a  key  component,  the  potential  benefits  of  new  classification 
composites  can  be  evaluated  under  approximately  real  world  classification  system  conditions. 
Ultimately  the  better  classification  composites  will  be  those  that  can  show  improvements  (e.g., 
lower  attrition  or  better  performance)  under  operational  conditions,  and  not  only  under  idealized 
research  conditions.  The  JCM  also  provides  a  unified  tool  for  managing  personnel  classification 
systems  beyond,  or  in  combination  with,  incentives.  For  example,  using  a  JCM  the  analyst  can 
examine  the  impact  of  cut  score  changes  on  two  or  more  MOS  simultaneously,  while  taking  into 
account  applicant  preferences.  This  is  in  contrast  to  the  current  practice  of  examining  one  MOS 
at  a  time,  which  disregards  the  impact  of  lowering/raising  the  cut  score  on  other  MOS  and 
applicant  preferences. 

Overall,  modeling  techniques,  such  as  discrete  choice  modeling,  provide  a  framework  for 
bringing  together  an  array  of  factors  (e.g.,  aptitude,  personality,  and  economic)  that  influence 
individuals’  job  choices  under  real  world  conditions.  Because  of  this,  these  techniques  constitute 
an  important  tool  and  one  that  should  have  a  more  prominent  role  in  personnel  classification 
research. 
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Appendix  A:  Model  Fit  Diagnostics 


A-l 


A-2 


Table  A-l.  Estimation  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Subgroup 

Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

ALL 

1 

1 IX 

2707 

2729 

-22 

1.01 

14.40 

14.51 

-0.12 

2 

13F 

349 

357 

-8 

1.02 

1.86 

1.90 

-0.04 

3 

FA1 

794 

752 

42 

0.95 

4.22 

4.00 

0.22 

4 

FA2 

197 

196 

1 

0.99 

1.05 

1.04 

0.01 

5 

ADI 

147 

153 

-6 

1.04 

0.78 

0.81 

-0.03 

6 

AVI 

264 

288 

-24 

1.09 

1.40 

1.53 

-0.13 

7 

AV2 

185 

171 

14 

0.92 

0.98 

0.91 

0.07 

8 

18X 

236 

220 

16 

0.93 

1.26 

1.17 

0.09 

9 

19D 

559 

555 

4 

0.99 

2.97 

2.95 

0.02 

10 

19K 

229 

234 

-5 

1.02 

1.22 

1.24 

-0.03 

II 

EN1 

339 

354 

-15 

1.04 

1.80 

1.88 

-0.08 

12 

EN2 

316 

320 

-4 

1.01 

1.68 

1.70 

-0.02 

13 

Sll 

290 

310 

-20 

1.07 

1.54 

1.65 

-0.11 

14 

SI2 

664 

668 

-4 

1.01 

3.53 

3.55 

-0.02 

15 

PA1 

40 

42 

-2 

1.04 

0.21 

0.22 

-0.01 

16 

LEI 

542 

515 

27 

0.95 

2.88 

2.74 

0.14 

17 

ELI 

137 

152 

-15 

1.11 

0.73 

0.81 

-0.08 

18 

EL2 

20 

21 

-1 

1.05 

0.11 

0.11 

-0.01 

19 

AX1 

66 

68 

-2 

1.03 

0.35 

0.36 

-0.01 

20 

AMI 

46 

49 

-3 

1.07 

0.24 

0.26 

-0.02 

21 

52D 

122 

124 

-2 

1.02 

0.65 

0.66 

-0.01 

22 

63B 

551 

575 

-24 

1.04 

2.93 

3.06 

-0.13 

23 

VM1 

275 

282 

-7 

1.03 

1.46 

1.50 

-0.04 

24 

VM2 

24 

26 

-2 

1.10 

0.13 

0.14 

-0.01 

25 

74D 

213 

231 

-18 

1.08 

1.13 

1.23 

-0.10 

26 

TR1 

108 

102 

6 

0.95 

0.57 

0.54 

0.03 

27 

88M 

377 

364 

13 

0.97 

2.00 

1.94 

0.07 

28 

89D 

110 

105 

5 

0.95 

0.59 

0.56 

0.03 

29 

89B 

196 

196 

1.00 

1.04 

1.04 

0.00 

30 

MD1 

978 

1016 

-38 

1.04 

5.20 

5.40 

-0.20 

31 

92F 

630 

621 

9 

0.99 

3.35 

3.30 

0.05 

32 

92G 

206 

202 

4 

0.98 

1.10 

1.07 

0.02 

33 

SL1 

437 

461 

-24 

1.06 

2.32 

2.45 

-0.13 

34 

INI 

543 

542 

1 

1.00 

2.89 

2.88 

0.01 

35 

Fill 

600 

623 

-23 

1.04 

3.19 

3.31 

-0.12 

36 

98X 

99 

102 

-3 

1.03 

0.53 

0.54 

-0.02 

999 

Non-Acc 

5207 

5076 

131 

0.97 

27.69 

27.00 

0.70 

MALE 

1 

1 IX 

2707 

2729 

-22 

1.01 

18.02 

18.16 

-0.14 

2 

13F 

349 

357 

-8 

1.02 

2.32 

2.38 

-0.05 

3 

FA1 

794 

752 

42 

0.95 

5.28 

5.00 

0.28 

4 

FA2 

186 

160 

27 

0.86 

1.24 

1.06 

0.17 

5 

ADI 

125 

108 

17 

0.87 

0.83 

0.72 

0.11 

6 

AVI 

234 

224 

10 

0.96 

1.56 

1.49 

0.07 

7 

AV2 

131 

144 

-13 

1.10 

0.87 

0.96 

-0.09 

8 

18X 

236 

220 

16 

0.93 

1.57 

1.46 

0.11 

9 

19D 

559 

555 

4 

0.99 

3.72 

3.69 

0.03 

10 

19K 

229 

234 

-5 

1.02 

1.52 

1.56 

-0.03 

11 

EN1 

300 

292 

8 

0.97 

2.00 

1.94 

0.05 

12 

EN2 

271 

296 

-25 

1.09 

1.80 

1.97 

-0.17 

13 

Sll 

146 

151 

-5 

1.03 

0.97 

1.00 

-0.03 

14 

SI2 

468 

518 

-50 

1.11 

3.12 

3.45 

-0.33 

15 

PA1 

25 

24 

1 

0.95 

0.17 

0.16 

0.01 

16 

LEI 

373 

379 

-6 

1.02 

2.48 

2.52 

-0.04 

17 

ELI 

128 

141 

-13 

1.10 

0.85 

0.94 

-0.09 

18 

EL2 

17 

19 

-2 

1.14 

0.11 

0.13 

-0.01 

19 

AX1 

46 

45 

0.99 

0.31 

0.30 

0.01 

A-3 


Table  A-l.  Estimation  Sample  MOS Fit  Diagnostics  by  Subgroup 


Subgroup 

Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

20 

AMI 

36 

39 

-3 

1.08 

0.24 

0.26 

-0.02 

21 

52D 

75 

108 

-33 

1.45 

0.50 

0.72 

-0.22 

22 

63  B 

533 

512 

22 

0.96 

3.55 

3.41 

0.14 

23 

VM1 

248 

265 

-16 

1.07 

1.65 

1.76 

-0.11 

24 

VM2 

24 

26 

-2 

1.10 

0.16 

0.17 

-0.01 

25 

74D 

103 

113 

-10 

1.10 

0.69 

0.75 

-0.07 

26 

TR1 

68 

56 

12 

0.82 

0.45 

0.37 

0.08 

27 

88M 

229 

234 

-5 

1.02 

1.52 

1.56 

-0.03 

28 

89D 

100 

93 

7 

0.93 

0.67 

0.62 

0.05 

29 

89B 

71 

92 

-21 

1.29 

0.47 

0.61 

-0.14 

30 

MD1 

649 

664 

-14 

1.02 

4.32 

4.42 

-0.10 

31 

92F 

449 

428 

21 

0.95 

2.99 

2.85 

0.14 

32 

92G 

136 

115 

21 

0.84 

0.91 

0.77 

0.14 

33 

SL1 

233 

239 

-6 

1.02 

1.55 

1.59 

-0.04 

34 

INI 

454 

457 

-2 

1.01 

3.02 

3.04 

-0.02 

35 

Fill 

463 

450 

13 

0.97 

3.08 

2.99 

0.09 

36 

98X 

70 

64 

6 

0.92 

0.47 

0.43 

0.04 

999 

Non-Acc 

3759 

3724 

36 

0.99 

25.02 

24.78 

0.24 

FEMALE 

1 

1 IX 

2 

13F 

3 

FA1 

4 

FA2 

1 1 

36 

-25 

3.33 

0.29 

0.95 

-0.66 

5 

ADI 

22 

44 

-23 

2.05 

0.58 

1.16 

-0.58 

6 

AVI 

30 

64 

-34 

2.13 

0.79 

1.69 

-0.90 

7 

AV2 

54 

27 

27 

0.50 

1.43 

0.71 

0.71 

8 

18X 

9 

19D 

10 

19K 

1 1 

F.N1 

39 

62 

-23 

1.60 

1.03 

1.64 

-0.61 

12 

EN2 

45 

24 

21 

0.53 

1.19 

0.64 

0.56 

13 

Sll 

144 

160 

-16 

1.11 

3.81 

4.24 

-0.42 

14 

S12 

196 

150 

45 

0.77 

5.19 

3.97 

1.22 

15 

PA1 

15 

18 

-3 

1.18 

0.40 

0.48 

-0.08 

16 

LEI 

169 

136 

33 

0.81 

4.47 

3.60 

0.87 

17 

ELI 

9 

12 

-3 

1.29 

0.24 

0.32 

-0.08 

18 

EL2 

3 

2 

1 

0.53 

0.08 

0.05 

0.03 

19 

AX1 

20 

23 

-2 

1.12 

0.53 

0.61 

-0.08 

20 

AMI 

10 

10 

1.02 

0.26 

0.26 

0.00 

21 

52D 

47 

16 

31 

0.34 

1.24 

0.42 

0.82 

22 

63  B 

18 

64 

-46 

3.58 

0.48 

1.69 

-1.22 

23 

VM1 

27 

17 

9 

0.65 

0.71 

0.45 

0.26 

24 

VM2 

25 

74D 

110 

118 

-8 

1.07 

2.91 

3.12 

-0.21 

26 

TR1 

40 

46 

-6 

1.15 

1.06 

1.22 

-0.16 

27 

88M 

148 

130 

18 

0.88 

3.92 

3.44 

0.48 

28 

89D 

10 

12 

-2 

1.15 

0.26 

0.32 

-0.05 

29 

89B 

125 

104 

20 

0.84 

3.31 

2.75 

0.55 

30 

MD1 

329 

353 

-24 

1.07 

8.71 

9.34 

-0.64 

31 

92F 

181 

193 

-12 

1.06 

4.79 

5.11 

-0.32 

32 

92G 

70 

87 

-17 

1.24 

1.85 

2.30 

-0.45 

33 

SL1 

204 

222 

-19 

1.09 

5.40 

5.88 

-0.48 

34 

INI 

89 

85 

4 

0.96 

2.36 

2.25 

0.11 

35 

Fill 

137 

173 

-35 

1.26 

3.63 

4.58 

-0.95 

36 

98X 

29 

38 

-9 

1.30 

0.77 

1.01 

-0.24 

999 

Non-Acc 

1448 

1352 

96 

0.93 

38.32 

35.79 

2.53 

Table  A-2.  Estimation  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Subgroup 

Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

HGC 

1 

1 IX 

1536 

1504 

32 

0.98 

12.54 

12.28 

0.26 

2 

13F 

205 

185 

20 

0.90 

1.67 

1.51 

0.16 

3 

FA1 

498 

465 

33 

0.93 

4.07 

3.80 

0.27 

4 

FA2 

75 

110 

-35 

1.47 

0.61 

0.90 

-0.29 

5 

ADI 

80 

78 

2 

0.97 

0.65 

0.64 

0.02 

6 

AVI 

174 

166 

8 

0.95 

1.42 

1.36 

0.07 

7 

AV2 

121 

126 

-5 

1.04 

0.99 

1.03 

-0.04 

8 

18X 

198 

184 

14 

0.93 

1.62 

1.50 

0.11 

9 

19D 

383 

388 

-5 

1.01 

3.13 

3.17 

-0.04 

10 

19K. 

142 

162 

-20 

1.14 

1.16 

1.32 

-0.16 

I! 

EN1 

264 

253 

11 

0.96 

2.16 

2.07 

0.09 

12 

EN2 

223 

241 

-18 

1.08 

1.82 

1.97 

-0.15 

13 

SI1 

213 

223 

-10 

1.05 

1.74 

1.82 

-0.08 

14 

S12 

475 

449 

26 

0.95 

3.88 

3.67 

0.21 

15 

PA1 

32 

38 

-5 

1.16 

0.26 

0.31 

-0.05 

16 

LEI 

383 

411 

-28 

1.07 

3.13 

3.36 

-0.23 

17 

ELI 

105 

114 

-9 

1.08 

0.86 

0.93 

-0.07 

18 

EL2 

18 

19 

-2 

1.09 

0.15 

0.16 

-0.01 

19 

AX  1 

47 

49 

-2 

1.04 

0.38 

0.40 

-0.02 

20 

AMI 

25 

27 

-2 

1.09 

0.20 

0.22 

-0.02 

21 

52D 

82 

80 

2 

0.97 

0.67 

0.65 

0.02 

22 

63  B 

379 

376 

3 

0.99 

3.10 

3.07 

0.02 

23 

VM1 

158 

178 

-21 

1.13 

1.29 

1.45 

-0.16 

24 

VM2 

11 

10 

1 

0.93 

0.09 

0.08 

0.01 

25 

74D 

161 

171 

-10 

1.06 

1.31 

1.40 

-0.08 

26 

TR1 

78 

72 

6 

0.92 

0.64 

0.59 

0.05 

27 

88M 

293 

263 

31 

0.89 

2.39 

2.15 

0.24 

28 

89D 

69 

91 

-23 

1.33 

0.56 

0.74 

-0.18 

29 

89B 

117 

136 

-19 

1.17 

0.96 

1.1 1 

-0.16 

30 

MD1 

755 

779 

-23 

1.03 

6.17 

6.36 

-0.20 

31 

92F 

526 

492 

34 

0.94 

4.30 

4.02 

0.28 

32 

92  G 

160 

161 

-1 

1.01 

1.31 

1.31 

-0.01 

33 

SL1 

719 

343 

-24 

1.07 

2.61 

2.80 

-0.20 

34 

INI 

415 

423 

-9 

1.02 

3.39 

3.45 

-0.07 

35 

mi 

509 

432 

78 

0.85 

4.16 

3.53 

0.63 

36 

98X 

90 

94 

-4 

1.05 

0.73 

0.77 

-0.03 

999 

Non-Acc 

2926 

2952 

-26 

1.01 

23.90 

24.11 

-0.21 

SENIOR 

1 

1 IX 

597 

734 

-137 

1.23 

15.76 

19.35 

-3.58 

2 

13F 

24 

55 

-31 

2.32 

0.63 

1.45 

-0.82 

3 

FA1 

30 

94 

-64 

3.13 

0.79 

2.48 

-1.69 

4 

FA2 

6 

22 

-15 

3.53 

0.16 

0.58 

-0.42 

5 

ADI 

12 

27 

-15 

2.22 

0.32 

0.71 

-0.39 

6 

AVI 

75 

61 

14 

0.81 

1.98 

1.61 

0.37 

7 

AV2 

44 

29 

15 

0.66 

1.16 

0.76 

0.40 

8 

18X 

38 

35 

3 

0.91 

1.00 

0.92 

0.08 

9 

19D 

77 

73 

4 

0.95 

2.03 

1.92 

0.11 

10 

19K 

40 

30 

10 

0.74 

1.06 

0.79 

0.27 

11 

EN1 

24 

39 

-15 

1.63 

0.63 

1.03 

-0.39 

12 

EN2 

33 

43 

-9 

1.27 

0.87 

1.13 

-0.26 

13 

SI1 

57 

59 

-2 

1.04 

1.5! 

1.56 

-0.05 

14 

S12 

96 

106 

-10 

1.10 

2.53 

2.79 

-0.26 

15 

PA1 

6 

3 

4 

0.42 

0.16 

0.08 

0.08 

16 

LEI 

138 

58 

80 

0.42 

3.64 

1.53 

2.12 

17 

ELI 

8 

10 

-1 

1.18 

0.21 

0.26 

-0.05 

18 

EL2 

1 

I 

1.04 

0.03 

0.03 

0.00 

19 

AX1 

19 

13 

6 

0.67 

0.50 

0.34 

0.16 

A -5 


Table  A-2.  Estimation  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Subgroup 

Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

20 

AMI 

16 

12 

4 

0.73 

0.42 

0.32 

0.11 

21 

52D 

21 

24 

-4 

1.17 

0.55 

0.63 

-0.08 

22 

63B 

103 

125 

-22 

1.21 

2.72 

3.29 

-0.57 

23 

VM1 

21 

31 

-10 

1.47 

0.55 

0.82 

-0.26 

24 

VM2 

1 1 

14 

-3 

1.27 

0.29 

0.37 

-0.08 

25 

74  D 

31 

37 

-5 

1.17 

0.82 

0.98 

-0.16 

26 

TR! 

16 

17 

-1 

1.05 

0.42 

0.45 

-0.03 

27 

88M 

14 

46 

-32 

3.27 

0.37 

1.21 

-0.84 

28 

89D 

21 

4 

17 

0.18 

0.55 

0.11 

0.45 

29 

89B 

21 

17 

5 

0.79 

0.55 

0.45 

0.11 

30 

MD1 

173 

173 

-1 

1.00 

4.57 

4.56 

0.01 

31 

92  F 

14 

30 

-15 

2.05 

0.37 

0.79 

-0.42 

32 

92G 

24 

25 

-1 

1.03 

0.63 

0.66 

-0.03 

33 

SL1 

78 

76 

2 

0.97 

2.06 

2.00 

0.06 

34 

INI 

127 

96 

31 

0.76 

3.35 

2.53 

0.82 

35 

HI1 

44 

76 

-31 

1.70 

1.16 

2.00 

-0.84 

36 

98X 

9 

8 

1 

0.87 

0.24 

0.21 

0.03 

999 

Non-Acc 

1718 

1491 

227 

0.87 

45.37 

39.30 

6.07 

NG 

1 

1 IX 

574 

491 

83 

0.86 

20.74 

17.75 

2.99 

2 

13F 

120 

117 

3 

0.97 

4.34 

4.23 

0.11 

3 

FA1 

266 

193 

73 

0.72 

9.61 

6.98 

2.64 

4 

FA2 

116 

64 

52 

0.55 

4.19 

2.31 

1.88 

5 

ADI 

55 

48 

7 

0.87 

1.99 

1.74 

0.25 

6 

AVI 

15 

61 

-46 

4.08 

0.54 

2.21 

-1.66 

7 

AV2 

20 

15 

4 

0.78 

0.72 

0.54 

0.18 

8 

18X 

1 

-1 

0.04 

-0.04 

9 

19D 

100 

95 

5 

0.95 

3.61 

3.43 

0.18 

10 

19K 

47 

42 

5 

0.90 

1.70 

1.52 

0.18 

1 1 

EN1 

51 

62 

-1  1 

1.22 

1.84 

2.24 

-0.40 

12 

EN2 

59 

36 

23 

0.61 

2.13 

1.30 

0.83 

13 

Sll 

20 

28 

-8 

1.41 

0.72 

1.01 

-0.29 

14 

S12 

93 

113 

-20 

1.22 

3.36 

4.09 

-0.72 

15 

PA  1 

1 

1 

1.23 

0.04 

0.04 

0.00 

16 

LF.l 

21 

46 

-25 

2.20 

0.76 

1.66 

-0.90 

17 

ELI 

24 

28 

-5 

1.21 

0.87 

1.01 

-0.14 

18 

EL.2 

1 

1 

0.33 

0.04 

0.04 

19 

AXI 

7 

-7 

0.25 

-0.25 

20 

AMI 

5 

10 

-5 

1.97 

0.18 

0.36 

-0.18 

21 

52D 

19 

20 

-1 

1.05 

0.69 

0.72 

-0.04 

22 

63B 

69 

75 

-6 

1.08 

2.49 

2.71 

-0.22 

23 

VMI 

96 

73 

23 

0.76 

3.47 

2.64 

0.83 

24 

VM2 

2 

2 

1.1 1 

0.07 

0.07 

0.00 

25 

74D 

21 

24 

-3 

1.13 

0.76 

0.87 

-0.11 

26 

TR1 

14 

13 

1 

0.94 

0.51 

0.47 

0.04 

27 

88M 

69 

55 

14 

0.80 

2.49 

1.99 

0.51 

28 

89D 

20 

10 

10 

0.49 

0.72 

0.36 

0.36 

29 

89B 

58 

43 

15 

0.75 

2.10 

1.55 

0.54 

30 

MD1 

50 

64 

-14 

1.28 

1.81 

2.31 

-0.51 

31 

92  F 

90 

100 

-10 

1.11 

3.25 

3.62 

-0.36 

32 

92G 

22 

16 

6 

0.73 

0.80 

0.58 

0.22 

33 

SL1 

40 

43 

-3 

1.08 

1.45 

1.55 

-0.11 

34 

INI 

1 

22 

-21 

16.25 

0.04 

0.80 

-0.76 

35 

HI1 

46 

1 15 

-69 

2.49 

1.66 

4.16 

-2.50 

36 

98X 

999 

Non-Acc 

562 

633 

-71 

1.13 

20.31 

22.89 

-2.57 

A-6 


Table  A- 3.  Estimation  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Subgroup 

Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

I-IIIA 

1 

1 IX 

1739 

1758 

-19 

1.01 

13.18 

13.32 

-0.14 

2 

13F 

258 

229 

29 

0.89 

1.95 

1.74 

0.22 

3 

FA1 

516 

537 

-21 

1.04 

3.91 

4.07 

-0.16 

4 

FA2 

129 

139 

-10 

1.08 

0.98 

1.05 

-0.08 

5 

ADI 

136 

136 

1.00 

1.03 

1.03 

0.00 

6 

AVI 

249 

249 

1.00 

1.89 

1.89 

0.00 

7 

AV2 

121 

137 

-16 

1.14 

0.92 

1.04 

-0.12 

8 

18X 

236 

220 

16 

0.93 

1.79 

1.67 

0.12 

9 

19D 

344 

356 

-13 

1.04 

2.61 

2.70 

-0.09 

10 

19K 

134 

141 

-8 

1.06 

1.02 

1.07 

-0.05 

11 

EN1 

166 

202 

-36 

1.22 

1.26 

1.53 

-0.27 

12 

EN2 

215 

197 

18 

0.92 

1.63 

1.49 

0.14 

13 

Sll 

222 

242 

-20 

1.09 

1.68 

1.83 

-0.15 

14 

S12 

518 

550 

-32 

1.06 

3.92 

4.17 

-0.24 

15 

PA1 

40 

42 

-2 

1.04 

0.30 

0.32 

-0.02 

16 

LEI 

336 

387 

-50 

1.15 

2.55 

2.93 

-0.39 

17 

ELI 

108 

117 

-8 

1.08 

0.82 

0.89 

-0.07 

18 

EL2 

17 

20 

-4 

1.21 

0.13 

0.15 

-0.02 

19 

AX1 

41 

45 

-4 

1.11 

0.31 

0.34 

-0.03 

20 

AMI 

34 

38 

-3 

1.10 

0.26 

0.29 

-0.03 

21 

52D 

65 

74 

-9 

1.14 

0.49 

0.56 

-0.07 

22 

63B 

350 

376 

-26 

1.07 

2.65 

2.85 

-0.20 

23 

VM1 

165 

161 

4 

0.98 

1.25 

1.22 

0.03 

24 

VM2 

12 

17 

-5 

1.39 

0.09 

0.13 

-0.04 

25 

74D 

136 

157 

-21 

1.15 

1.03 

1.19 

-0.16 

26 

TR1 

43 

38 

5 

0.88 

0.33 

0.29 

0.04 

27 

88M 

129 

165 

-36 

1.28 

0.98 

1.25 

-0.27 

28 

89D 

99 

99 

-1 

1.01 

0.75 

0.75 

0.00 

29 

89B 

70 

84 

-14 

1.20 

0.53 

0.64 

-0.11 

30 

MD1 

923 

958 

-36 

1.04 

6.99 

7.26 

-0.27 

31 

92F 

272 

249 

23 

0.92 

2.06 

1.89 

0.17 

32 

92G 

114 

89 

25 

0.78 

0.86 

0.67 

0.19 

33 

SL1 

193 

229 

-36 

1.19 

1.46 

1.74 

-0.27 

34 

INI 

526 

511 

16 

0.97 

3.99 

3.87 

0.11 

35 

HU 

559 

507 

52 

0.91 

4.24 

3.84 

0.39 

36 

98X 

99 

102 

-3 

1.03 

0.75 

0.77 

-0.02 

999 

Non-Acc 

3884 

3639 

246 

0.94 

29.43 

27.57 

1.85 

1IIB+IV 

1 

1 IX 

968 

971 

-3 

1.00 

17.27 

17.32 

-0.05 

2 

13F 

91 

128 

-37 

1.41 

1.62 

2.28 

-0.66 

3 

FA1 

278 

214 

64 

0.77 

4.96 

3.82 

1.14 

4 

FA2 

68 

57 

11 

0.84 

1.21 

1.02 

0.20 

5 

ADI 

11 

17 

-6 

1.56 

0.20 

0.30 

-0.11 

6 

AVI 

15 

39 

-24 

2.60 

0.27 

0.70 

-0.43 

7 

AV2 

64 

34 

30 

0.53 

1.14 

0.61 

0.54 

8 

18X 

9 

19D 

215 

199 

16 

0.92 

3.84 

3.55 

0.29 

10 

19K 

95 

93 

3 

0.97 

1.69 

1.66 

0.04 

11 

EN1 

173 

152 

21 

0.88 

3.09 

2.71 

0.38 

12 

EN2 

101 

123 

-22 

1.21 

1.80 

2.19 

-0.39 

13 

Sll 

68 

68 

1.00 

1.21 

1.21 

0.00 

14 

S12 

146 

118 

28 

0.81 

2.60 

2.10 

0.50 

15 

PA1 

16 

LEI 

206 

129 

77 

0.63 

3.68 

2.30 

1.37 

17 

ELI 

29 

36 

-7 

1.23 

0.52 

0.64 

-0.12 

18 

EL2 

3 

1 

3 

0.18 

0.05 

0.02 

0.04 

19 

AX1 

25 

23 

2 

0.91 

0.45 

0.41 

0.04 
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Table  A-3.  Estimation  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Subgroup 

Ah.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

20 

AMI 

12 

11 

0.97 

0.21 

0.20 

0.02 

21 

52D 

57 

51 

7 

0.88 

1.02 

0.91 

0.11 

22 

63B 

201 

199 

2 

0.99 

3.59 

3.55 

0.04 

23 

VM1 

110 

121 

-1 1 

1.10 

1.96 

2.16 

-0.20 

24 

VM2 

12 

9 

2 

0.80 

0.21 

0.16 

0.05 

25 

74D 

77 

75 

3 

0.96 

1.37 

1.34 

0.04 

26 

TR1 

65 

64 

1 

0.99 

1.16 

1.14 

0.02 

27 

88M 

248 

199 

49 

0.80 

4.42 

3.55 

0.88 

28 

89D 

11 

5 

6 

0.47 

0.20 

0.09 

0.11 

29 

89B 

126 

112 

14 

0.89 

2.25 

2.00 

0.25 

30 

MD1 

55 

58 

-3 

1.05 

0.98 

1.03 

-0.05 

31 

92  F 

358 

372 

-14 

1.04 

6.39 

6.63 

-0.25 

32 

92G 

92 

113 

-20 

1.22 

1.64 

2.02 

-0.37 

33 

SL1 

244 

233 

12 

0.95 

4.35 

4.16 

0.20 

34 

INI 

17 

31 

-14 

1.86 

0.30 

0.55 

-0.25 

35 

Hll 

41 

115 

-74 

2.82 

0.73 

2.05 

-1.32 

36 

98X 

999 

Non-Acc 

1323 

1437 

-114 

1.09 

23.60 

25.63 

-2.02 
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Table  A-4.  Estimation  Sample  TOS  Fit  Diagnostics  by  Subgroup 


Subgoup 

TOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

ALL 

3 

5765 

5730 

35 

0.99 

30.66 

30.47 

0.19 

4 

4844 

4945 

-101 

1.02 

25.76 

26.30 

-0.54 

5 

2128 

2169 

-4! 

1.02 

11.32 

11.53 

-0.22 

6 

859 

884 

-25 

1.03 

4.57 

4.70 

-0.13 

Non-Acc 

5207 

5076 

131 

0.97 

27.69 

26.99 

0.70 

MALE 

3 

5014 

4978 

36 

0.99 

33.37 

33.13 

0.24 

4 

3852 

3904 

-52 

1.01 

25.64 

25.98 

-0.34 

5 

1771 

1787 

-16 

1.01 

11.79 

11.89 

-0.11 

6 

629 

633 

-4 

1.01 

4.19 

4.21 

-0.03 

Non-Acc 

3759 

3724 

36 

0.99 

25.02 

24.78 

0.23 

FEMALE 

3 

751 

752 

-1 

1.00 

19.88 

19.91 

-0.03 

4 

992 

1041 

-49 

1.05 

26.26 

27.56 

-1.30 

5 

357 

382 

-25 

1.07 

9.45 

10.11 

-0.66 

6 

230 

250 

-20 

1.09 

6.09 

6.62 

-0.53 

Non-Acc 

1448 

1352 

96 

0.93 

38.33 

35.80 

2.53 

HGC 

3 

3556 

3500 

56 

0.98 

29.04 

28.58 

0.46 

4 

3597 

3627 

-30 

1.01 

29.38 

29.62 

-0.24 

5 

1579 

1576 

2 

1.00 

12.90 

12.87 

0.02 

6 

587 

590 

-3 

1.01 

4.79 

4.82 

-0.02 

Non-Acc 

2926 

2952 

-26 

1.01 

23.90 

24.11 

-0.21 

SENIOR 

3 

497 

952 

-455 

1.92 

13.11 

25.11 

-12.00 

4 

884 

778 

106 

0.88 

23.31 

20.52 

2.80 

5 

467 

397 

70 

0.85 

12.32 

10.47 

1.85 

6 

226 

174 

52 

0.77 

5.96 

4.59 

1.37 

Non-Acc 

1718 

1491 

227 

0.87 

45.31 

39.32 

5.99 

NG 

3 

1713 

1279 

434 

0.75 

61.91 

46.22 

15.68 

4 

363 

540 

-177 

1.49 

13.12 

19.52 

-6.40 

5 

82 

195 

-113 

2.37 

2.96 

7.05 

-4.08 

6 

47 

120 

-73 

2.57 

1.70 

4.34 

-2.64 

Non-Acc 

562 

633 

-71 

1.13 

20.31 

22.88 

-2.57 

I-IIIA 

3 

2533 

2580 

-47 

1.02 

19.19 

19.55 

-0.36 

4 

4116 

4241 

-125 

1.03 

31.19 

32.14 

-0.95 

5 

1892 

1939 

-47 

1.03 

14.34 

14.69 

-0.36 

6 

772 

798 

-26 

1.03 

5.85 

6.05 

-0.20 

Non-Acc 

3884 

3639 

246 

0.94 

29.43 

27.57 

1.86 

IIIB+IV 

3 

3232 

3150 

82 

0.97 

57.65 

56.19 

1.46 

4 

728 

704 

25 

0.97 

12.99 

12.56 

0.43 

5 

236 

229 

7 

0.97 

4.21 

4.08 

0.12 

6 

87 

86 

1 

0.98 

1.55 

1.53 

0.02 

Non-Acc 

1323 

1437 

-114 

1.09 

23.60 

25.63 

-2.03 
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Table  A-5.  Hold-Out  (Validation)  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Subgroup 

Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

ALL 

1 

1 IX 

2707 

2726 

-19 

1.01 

14.40 

14.50 

-0.10 

2 

13F 

349 

345 

4 

0.99 

1.86 

1.83 

0.02 

3 

FA1 

794 

766 

28 

0.96 

4.22 

4.07 

0.15 

4 

FA2 

197 

189 

8 

0.96 

1.05 

1.01 

0.04 

5 

ADI 

147 

151 

-4 

1.02 

0.78 

0.80 

-0.02 

6 

AVI 

264 

256 

8 

0.97 

1.40 

1.36 

0.04 

7 

AV2 

185 

166 

19 

0.90 

0.98 

0.88 

0.10 

8 

18X 

236 

226 

10 

0.96 

1.26 

1.20 

0.05 

9 

19D 

559 

567 

-8 

1.01 

2.97 

3.02 

-0.04 

10 

19K 

229 

253 

-24 

1.1 1 

1.22 

1.35 

-0.13 

11 

EN1 

339 

339 

1.00 

1.80 

1.80 

12 

EN2 

316 

345 

-29 

1.09 

1.68 

1.83 

-0.15 

13 

Sll 

290 

300 

-10 

1.04 

1.54 

1.60 

-0.05 

14 

SI2 

664 

642 

22 

0.97 

3.53 

3.41 

0.12 

15 

PA  1 

40 

33 

7 

0.81 

0.21 

0.18 

0.04 

16 

LEI 

542 

535 

7 

0.99 

2.88 

2.85 

0.04 

17 

ELI 

137 

154 

-17 

1.13 

0.73 

0.82 

-0.09 

18 

EL2 

20 

20 

0.98 

0.11 

0.11 

19 

AX1 

66 

80 

-14 

1.21 

0.35 

0.43 

-0.07 

20 

AMI 

46 

49 

-3 

1.07 

0.24 

0.26 

-0.02 

21 

52D 

122 

131 

-9 

1.07 

0.65 

0.70 

-0.05 

22 

63  B 

551 

574 

-23 

1.04 

2.93 

3.05 

-0.12 

23 

VM1 

275 

275 

1.00 

1.46 

1.46 

24 

VM2 

24 

22 

2 

0.91 

0.13 

0.12 

0.01 

25 

74D 

213 

247 

-34 

1.16 

1.13 

1.31 

-0.18 

26 

TR1 

108 

94 

14 

0.87 

0.57 

0.50 

0.07 

27 

88M 

377 

388 

-11 

1.03 

2.00 

2.06 

-0.06 

28 

89D 

110 

128 

-18 

1.17 

0.59 

0.68 

-0.10 

29 

8913 

196 

179 

17 

0.91 

1.04 

0.95 

0.09 

30 

MD1 

978 

976 

2 

1.00 

5.20 

5.19 

0.01 

31 

92F 

630 

642 

-12 

1.02 

3.35 

3.41 

-0.06 

32 

92G 

206 

186 

20 

0.90 

1.10 

0.99 

0.11 

33 

SL1 

437 

466 

-29 

1.07 

2.32 

2.48 

-0.15 

34 

INI 

543 

580 

-37 

1.07 

2.89 

3.08 

-0.20 

35 

HI1 

600 

624 

-24 

1.04 

3.19 

3.32 

-0.13 

36 

98X 

99 

100 

-1 

1.01 

0.53 

0.53 

-0.01 

999 

Non-Acc 

5207 

5049 

158 

0.97 

27.69 

26.85 

0.84 

MALF. 

1 

1 IX 

2706 

2725 

-19 

1.01 

17.72 

17.85 

-0.12 

2 

13F 

349 

345 

4 

0.99 

2.29 

2.26 

0.03 

3 

FA1 

794 

766 

28 

0.96 

5.20 

5.02 

0.18 

4 

FA2 

188 

159 

29 

0.85 

1.23 

1.04 

0.19 

5 

ADI 

123 

102 

21 

0.83 

0.81 

0.67 

0.14 

6 

AVI 

243 

205 

38 

0.84 

1.59 

1.34 

0.25 

7 

AV2 

142 

132 

10 

0.93 

0.93 

0.86 

0.07 

8 

18X 

236 

226 

10 

0.96 

1.55 

1.48 

0.07 

9 

19D 

559 

566 

-7 

1.01 

3.66 

3.71 

-0.05 

10 

19K 

229 

253 

-24 

1.1 1 

1.50 

1.66 

-0.16 

1 1 

EN1 

299 

291 

8 

0.97 

1.96 

1.91 

0.05 

12 

EN2 

298 

320 

-23 

1.08 

1.95 

2.10 

-0.14 

13 

Sll 

187 

161 

26 

0.86 

1.22 

1.05 

0.17 

14 

SI2 

494 

512 

-18 

1.04 

3.24 

3.35 

-0.12 

15 

PA1 

28 

18 

10 

0.66 

0.18 

0.12 

0.07 

16 

LEI 

399 

401 

-2 

1.01 

2.61 

2.63 

-0.01 

17 

ELI 

102 

136 

-34 

1.33 

0.67 

0.89 

-0.22 

18 

EL2 

20 

18 

2 

0.91 

0.13 

0.12 

0.01 

19 

AX1 

41 

50 

-9 

1.22 

0.27 

0.33 

-0.06 
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Table  AS.  Hold-Out  (Validation)  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Subgroup 

Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Difr.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

20 

AMI 

46 

41 

5 

0.89 

0.30 

0.27 

0.03 

21 

52D 

103 

115 

-11 

1.11 

0.67 

0.75 

-0.08 

22 

63B 

504 

527 

-23 

1.04 

3.30 

3.45 

-0.15 

23 

VM1 

257 

257 

1.00 

1.68 

1.68 

24 

VM2 

24 

22 

2 

0.91 

0.16 

0.14 

0.01 

25 

74D 

112 

116 

-4 

1.04 

0.73 

0.76 

-0.03 

26 

TR1 

62 

48 

14 

0.77 

0.41 

0.31 

0.09 

27 

88M 

276 

266 

11 

0.96 

1.81 

1.74 

0.07 

28 

89D 

102 

113 

-11 

1.11 

0.67 

0.74 

-0.07 

29 

89B 

101 

91 

10 

0.90 

0.66 

0.60 

0.07 

30 

MD1 

578 

613 

-35 

1.06 

3.79 

4.02 

-0.23 

31 

92  F 

476 

479 

-3 

1.01 

3.12 

3.14 

-0.02 

32 

92G 

131 

108 

24 

0.82 

0.86 

0.71 

0.15 

33 

SL1 

226 

254 

-28 

1.13 

1.48 

1.66 

-0.18 

34 

INI 

394 

490 

-96 

1.24 

2.58 

3.21 

-0.63 

35 

HI1 

441 

482 

-41 

1.09 

2.89 

3.16 

-0.27 

36 

98X 

71 

71 

1.00 

0.47 

0.47 

999 

Non-Acc 

3926 

3788 

138 

0.96 

25.72 

24.81 

0.90 

FEMALE 

1 

1 IX 

2 

13F 

3 

FA1 

4 

FA2 

9 

30 

-21 

3.36 

0.25 

0.85 

-0.60 

5 

ADI 

24 

48 

-24 

2.01 

0.68 

1.36 

-0.68 

6 

AVI 

21 

51 

-30 

2.41 

0.59 

1.44 

-0.85 

7 

AV2 

43 

34 

8 

0.80 

1.22 

0.96 

0.25 

8 

18X 

9 

I9D 

10 

19K 

11 

EN1 

40 

48 

-8 

1.20 

1.13 

1.36 

-0.23 

12 

EN2 

18 

24 

-6 

1.33 

0.51 

0.68 

-0.17 

13 

SI1 

103 

139 

-36 

1.35 

2.91 

3.94 

-1.02 

14 

S12 

170 

130 

40 

0.77 

4.81 

3.68 

1.13 

15 

PA1 

12 

14 

-2 

1.18 

0.34 

0.40 

-0.06 

16 

LEI 

143 

134 

9 

0.94 

4.04 

3.79 

0.25 

17 

ELI 

35 

18 

17 

0.52 

0.99 

0.51 

0.48 

18 

EL2 

1 

-i 

0.03 

-0.03 

19 

AX1 

25 

29 

-5 

1.19 

0.71 

0.82 

-0.11 

20 

AMI 

8 

-8 

0.23 

-0.23 

21 

52D 

19 

16 

3 

0.86 

0.54 

0.45 

0.08 

22 

63B 

47 

47 

1.00 

1.33 

1.33 

0.00 

23 

VM1 

18 

18 

1.00 

0.51 

0.51 

0.00 

24 

VM2 

25 

74D 

101 

131 

-30 

1.29 

2.86 

3.71 

-0.85 

26 

TR1 

46 

46 

1.00 

1.30 

1.30 

0.00 

27 

88M 

101 

122 

-22 

1.22 

2.86 

3.46 

-0.60 

28 

89D 

8 

15 

-7 

1.96 

0.23 

0.42 

-0.20 

29 

89B 

95 

88 

7 

0.93 

2.69 

2.49 

0.19 

30 

MD1 

400 

363 

37 

0.91 

11.31 

10.28 

1.03 

31 

92  F 

154 

163 

-9 

1.06 

4.36 

4.62 

-0.26 

32 

92G 

75 

79 

-4 

1.05 

2.12 

2.24 

-0.12 

33 

SL1 

211 

212 

-1 

1.00 

5.97 

6.00 

-0.04 

34 

INI 

149 

90 

59 

0.60 

4.21 

2.55 

1.66 

35 

HI  1 

159 

142 

18 

0.89 

4.50 

4.02 

0.48 

36 

98X 

28 

29 

-1 

1.05 

0.79 

0.82 

-0.03 

999 

Non-Acc 

1281 

1261 

20 

0.98 

36.23 

35.71 

0.52 

A-l  1 


Table  A-6.  Hold-Out  (Validation)  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

HOC 

1 

1 IX 

1716 

1600 

1 16 

0.93 

14.07 

13.12 

0.95 

2 

13F 

175 

167 

8 

0.96 

1.44 

1.37 

0.07 

3 

FA1 

490 

466 

24 

0.95 

4.02 

3.82 

0.20 

4 

FA2 

67 

108 

-41 

1.62 

0.55 

0.89 

-0.34 

5 

ADI 

88 

83 

5 

0.94 

0.72 

0.68 

0.04 

6 

AVI 

169 

152 

17 

0.90 

1.39 

1.25 

0.14 

7 

AV2 

128 

118 

10 

0.92 

1.05 

0.97 

0.08 

8 

18X 

195 

189 

5 

0.97 

1.60 

1.55 

0.05 

9 

19D 

366 

372 

-6 

1.02 

3.00 

3.05 

-0.05 

10 

19K 

145 

173 

-27 

1.19 

1.19 

1.42 

-0.23 

11 

EN1 

246 

238 

8 

0.97 

2.02 

1.95 

0.07 

12 

EN2 

223 

259 

-36 

1.16 

1.83 

2.12 

-0.29 

13 

SI1 

217 

203 

15 

0.93 

1.78 

1.66 

0.12 

14 

SI2 

481 

430 

51 

0.89 

3.94 

3.53 

0.42 

15 

PA  1 

28 

28 

1.00 

0.23 

0.23 

0.00 

16 

LEI 

361 

428 

-67 

1.19 

2.96 

3.51 

-0.55 

17 

ELI 

62 

104 

-42 

1.68 

0.51 

0.85 

-0.34 

18 

EL2 

18 

-18 

0.15 

-0.15 

19 

AX1 

48 

60 

-12 

1.25 

0.39 

0.49 

-0.10 

20 

AMI 

46 

29 

17 

0.63 

0.38 

0.24 

0.14  * 

21 

52D 

95 

79 

16 

0.83 

0.78 

0.65 

0.13 

22 

63  B 

365 

350 

15 

0.96 

2.99 

2.87 

0.12 

23 

VMI 

163 

187 

-24 

1.15 

1.34 

1.53 

-0.20 

24 

VM2 

6 

4 

2 

0.63 

0.05 

0.0.7 

0.02 

25 

74  D 

183 

189 

-6 

1.03 

1.50 

1.55 

-0.05 

26 

TR1 

76 

61 

15 

0.81 

0.62 

0.50 

0.12 

27 

88M 

268 

260 

8 

0.97 

2.20 

2.13 

0.07 

28 

89D 

67 

99 

-32 

1.47 

0.55 

0.81 

-0.26 

29 

89B 

136 

124 

12 

0.91 

1.12 

1.02 

0.10 

30 

MD1 

751 

720 

31 

0.96 

6.16 

5.90 

0.26 

31 

92  F 

516 

508 

8 

0.98 

4.23 

4.17 

0.07 

32 

92C> 

175 

138 

38 

0.79 

1.44 

1.13 

0.30 

33 

SL1 

333 

337 

-4 

1.01 

2.73 

2.76 

-0.03 

34 

INI 

393 

456 

-63 

1.16 

3.22 

3.74 

-0.52 

35 

Fill 

459 

452 

7 

0.99 

3.76 

3.71 

0.06 

36 

98X 

90 

85 

5 

0.94 

0.74 

0.70 

0.04 

999 

Non-Acc 

2867 

2922 

-55 

1.02 

23.51 

23.96 

-0.45 

SENIOR 

1 

1 IX 

533 

694 

-161 

1.30 

13.52 

17.63 

-4.10 

2 

13F 

40 

64 

-24 

1.60 

1.01 

1.63 

-0.61 

3 

FAI 

36 

105 

-68 

2.89 

0.91 

2.67 

-1.75 

4 

FA2 

7 

19 

-12 

2.68 

0.18 

0.48 

-0.30 

5 

ADI 

15 

31 

-15 

2.00 

0.38 

0.79 

-0.41 

6 

AVI 

89 

62 

27 

0.70 

2.26 

1.57 

0.68 

7 

AV2 

42 

33 

9 

0.79 

1.07 

0.84 

0.23 

8 

18X 

41 

36 

5 

0.88 

1.04 

0.91 

0.13 

9 

19D 

85 

97 

-12 

1.14 

2.16 

2.46 

-0.31 

10 

19K 

36 

39 

-3 

1.10 

0.91 

0.99 

-0,08 

1 1 

ENI 

50 

44 

5 

0.89 

1.27 

1.12 

0.15 

12 

F.N2 

56 

47 

9 

0.83 

1.42 

1.19 

0.23 

13 

SI1 

58 

63 

-6 

1.10 

1.47 

1.60 

-0.13 

14 

SI2 

82 

102 

-20 

1.24 

2.08 

2.59 

-0.51 

15 

PA1 

12 

3 

9 

0.25 

0.30 

0.08 

0.23 

16 

LEI 

158 

60 

97 

0.38 

4.01 

1.52 

2.49 

17 

ELI 

12 

1 1 

0.97 

0.30 

0.28 

0.03 

18 

EL2 

1 

-1 

0.03 

-0.03 

19 

AX  1 

16 

16 

0.99 

0.41 

0.41 

0.00 

A-12 


Table  A-6.  Hold-Out  (Validation)  Sample  MOS Fit  Diagnostics  by  Subgroup 


Subgroup 

Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

20 

AMI 

12 

-12 

0.30 

-0.30 

21 

52D 

6 

28 

-23 

4.93 

0.15 

0.71 

-0.56 

22 

63B 

128 

150 

-22 

1.17 

3.25 

3.81 

-0.56 

23 

VM1 

20 

33 

-13 

1.64 

0.51 

0.84 

-0.33 

24 

VM2 

18 

16 

2 

0.92 

0.46 

0.41 

0.05 

25 

74D 

16 

34 

-18 

2.14 

0.41 

0.86 

-0.46 

26 

TR1 

19 

17 

2 

0.89 

0.48 

0.43 

0.05 

27 

88M 

28 

54 

-26 

1.92 

0.71 

1.37 

-0.66 

28 

89D 

20 

9 

10 

0.48 

0.51 

0.23 

0.28 

29 

89B 

7 

15 

-8 

2.04 

0.18 

0.38 

-0.20 

30 

MD1 

176 

176 

1.00 

4.47 

4.47 

0.00 

31 

92F 

21 

35 

-14 

1.70 

0.53 

0.89 

-0.36 

32 

92G 

20 

35 

-14 

1.70 

0.51 

0.89 

-0.38 

33 

SL1 

64 

74 

-10 

1.16 

1.62 

1.88 

-0.26 

34 

INI 

136 

101 

35 

0.75 

3.45 

2.57 

0.89 

35 

Hll 

82 

82 

1.00 

2.08 

2.08 

0.00 

36 

98X 

6 

11 

-5 

1.83 

0.15 

0.28 

-0.13 

999 

Non-Acc 

1806 

1528 

278 

0.85 

45.83 

38.81 

7.01 

NG 

1 

11X 

458 

432 

26 

0.94 

17.18 

16.20 

0.98 

2 

13F 

135 

115 

20 

0.85 

5.06 

4.31 

0.75 

3 

FA1 

268 

196 

72 

0.73 

10.05 

7.35 

2.70 

4 

FA2 

123 

61 

62 

0.50 

4.61 

2.29 

2.33 

5 

ADI 

44 

37 

7 

0.85 

1.65 

1.39 

0.26 

6 

AVI 

6 

42 

-36 

6.71 

0.23 

1.57 

-1.35 

7 

AV2 

15 

15 

1.01 

0.56 

0.56 

0.00 

8 

18X 

1 

-1 

0.04 

-0.04 

9 

19D 

107 

98 

10 

0.91 

4.01 

3.67 

0.34 

10 

19K 

48 

41 

7 

0.86 

1.80 

1.54 

0.26 

11 

EN1 

43 

56 

-13 

1.31 

1.61 

2.10 

-0.49 

12 

EN2 

37 

39 

-2 

1.06 

1.39 

1.46 

-0.07 

13 

SI1 

15 

34 

-19 

2.28 

0.56 

1.27 

-0.71 

14 

S12 

100 

1 10 

-10 

1.10 

3.75 

4.12 

-0.37 

15 

PA1 

1 

-1 

0.04 

-0.04 

16 

LEI 

24 

47 

-23 

.,1.98 

0.90 

1.76 

-0.86 

17 

ELI 

64 

39 

24 

0.62 

2.40 

1.46 

0.94 

18 

EL2 

20 

1 

19 

0.03 

0.75 

0.04 

0.71 

19 

AX  1 

2 

4 

-2 

1.86 

0.08 

0.15 

-0.07 

20 

AMI 

9 

-9 

0.34 

-0.34 

21 

52D 

22 

23 

-2 

1.09 

0.83 

0.86 

-0.04 

22 

63  B 

57 

73 

-16 

1.28 

2.14 

2.74 

-0.60 

23 

VM1 

92 

55 

37 

0.60 

3.45 

2.06 

1.39 

24 

VM2 

2 

-2 

0.07 

-0.07 

25 

74D 

14 

24 

-10 

1.72 

0.53 

0.90 

-0.37 

26 

TR1 

13 

15 

-3 

1.20 

0.49 

0.56 

-0.07 

27 

88M 

81 

74 

7 

0.92 

3.04 

2.77 

0.26 

28 

89D 

23 

20 

4 

0.85 

0.86 

0.75 

0.11 

29 

89B 

52 

40 

13 

0.76 

1.95 

1.50 

0.45 

30 

MD1 

51 

79 

-28 

1.54 

1.91 

2.96 

-1.05 

31 

92F 

93 

99 

-6 

1.06 

3.49 

3.71 

-0.22 

32 

92G 

10 

14 

-4 

1.37 

0.38 

0.52 

-0.15 

33 

SL1 

40 

55 

-15 

1.37 

1.50 

2.06 

-0.56 

34 

INI 

14 

23 

-9 

1.63 

0.53 

0.86 

-0.34 

35 

Hll 

59 

90 

-31 

1.52 

2.21 

3.37 

-1.16 

36 

98X 

3 

4 

-1 

1.37 

0.11 

0.15 

-0.04 

999 

Non-Acc 

533 

599 

-66 

1.12 

19.99 

22.46 

-2.47 

A-13 


Table  A-7.  Hold-Out  (Validation)  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Subgroup 

Alt.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

I-I1IA 

1 

I IX 

1821 

1850 

-29 

1.02 

13.39 

13.60 

-0.21 

2 

13F 

237 

236 

1 

1.00 

1.74 

1.74 

0.01 

3 

FA1 

519 

568 

-49 

1.09 

3.82 

4.18 

-0.36 

4 

FA2 

151 

141 

10 

0.94 

1.11 

1.04 

0.07 

5 

ADI 

135 

133 

2 

0.99 

0.99 

0.98 

0.01 

6 

AVI 

236 

227 

9 

0.96 

1.74 

1.67 

0.07 

7 

AV2 

131 

136 

-5 

1.04 

0.96 

1.00 

-0.04 

8 

18X 

236 

226 

10 

0.96 

1.74 

1.66 

0.07 

9 

19D 

352 

364 

-12 

1.03 

2.59 

2.68 

-0.09 

10 

19K 

139 

162 

-23 

1.16 

1.02 

1.19 

-0.17 

1 1 

EN1 

190 

203 

-13 

1.07 

1.40 

1.49 

-0.10 

12 

EN2 

182 

214 

-32 

1.18 

1.34 

1.57 

-0.24 

13 

SI1 

244 

240 

4 

0.98 

1.79 

1.76 

0.03 

14 

SI2 

542 

543 

-1 

1.00 

3.99 

3.99 

-0.01 

15 

PA1 

40 

33 

7 

0.81 

0.29 

0.24 

0.05 

16 

LEI 

373 

401 

-28 

1.07 

2.74 

2.95 

-0.21 

17 

ELI 

127 

127 

1.00 

0.93 

0.93 

0.00 

18 

EL2 

20 

19 

1 

0.96 

0.15 

0.14 

0.01 

19 

AX  1 

36 

54 

-18 

1.49 

0.26 

0.40 

-0.13 

20 

AMI 

46 

38 

8 

0.83 

0.34 

0.28 

0.06 

21 

52D 

66 

80 

-14 

1.22 

0.49 

0.59 

-0.10 

22 

63B 

357 

383 

-26 

1.07 

2.63 

2.82 

-0.19 

23 

VM1 

157 

156 

2 

0.99 

1.15 

1.15 

0.01 

24 

VM2 

24 

17 

7 

0.70 

0.18 

0.13 

0.05 

25 

74D 

131 

165 

-34 

1.26 

0.96 

1.21 

-0.25 

26 

TR1 

62 

43 

19 

0.69 

0.46 

0.32 

0.14 

27 

88M 

176 

207 

-30 

1.17 

1.29 

1.52 

-0.23 

28 

89D 

105 

124 

-19 

1.18 

0.77 

0.91 

-0.14 

29 

89B 

71 

80 

-8 

1.12 

0.52 

0.59 

-0.07 

30 

MD1 

928 

929 

-1 

1.00 

6.82 

6.83 

-0.01 

31 

92F 

271 

264 

6 

0.98 

1.99 

1.94 

0.05 

32 

92G 

117 

96 

21 

0.82 

0.86 

0.71 

0.15 

33 

SL1 

212 

238 

-26 

1.12 

1.56 

1.75 

-0.19 

34 

INI 

525 

551 

-26 

1.05 

3.86 

4.05 

-0.19 

35 

HU 

562 

532 

30 

0.95 

4.13 

3.91 

0.22 

36 

98X 

93 

97 

-4 

1.04 

0.68 

0.71 

-0.03 

999 

Non-Acc 

3985 

3723 

261 

0.93 

29.30 

27.38 

1.93 

IIIBHV 

1 

11X 

886 

876 

10 

0.99 

17.02 

16.84 

0.19 

2 

13F 

1 13 

1 10 

3 

0.98 

2.17 

2.11 

0.06 

3 

FA1 

275 

198 

76 

0.72 

5.28 

3.81 

1.48 

4 

FA2 

46 

48 

-1 

1.03 

0.88 

0.92 

-0.04 

5 

ADI 

12 

17 

-5 

1.44 

0.23 

0.33 

-0.10 

6 

AVI 

28 

29 

-1 

1.05 

0.54 

0.56 

-0.02 

7 

AV2 

54 

30 

23 

0.57 

1.04 

0.58 

0.46 

8 

18X 

9 

19D 

207 

203 

4 

0.98 

3.98 

3.90 

0.08 

10 

19K 

90 

92 

-2 

1.02 

1.73 

1.77 

-0.04 

11 

EN1 

149 

135 

13 

0.91 

2.86 

2.59 

0.27 

12 

EN2 

134 

130 

4 

0.97 

2.57 

2.50 

0.08 

13 

SI1 

46 

60 

-14 

1.30 

0.88 

1.15 

-0.27 

14 

S12 

122 

99 

23 

0.81 

2.34 

1.90 

0.44 

15 

PA1 

16 

LEI 

169 

135 

34 

0.80 

3.25 

2.59 

0.65 

17 

ELI 

10 

27 

-17 

2.71 

0.19 

0.52 

-0.33 

18 

EL2 

19 

AX1 

30 

26 

4 

0.87 

0.58 

0.50 

0.08 

A-14 


Table  A-7.  Hold-Out  (Validation)  Sample  MOS  Fit  Diagnostics  by  Subgroup 


Subgroup 

AH.  ID 

MOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

20 

AMI 

11 

-11 

0.21 

-0.21 

21 

52D 

56 

50 

6 

0.90 

1.08 

0.96 

0.11 

22 

63B 

194 

191 

3 

0.98 

3.73 

3.67 

0.06 

23 

VM1 

118 

119 

-2 

1.01 

2.27 

2.29 

-0.02 

24 

VM2 

5 

-5 

0.10 

-0.10 

25 

74D 

82 

82 

1.00 

1.58 

1.58 

0.00 

26 

TR1 

46 

51 

-5 

1.11 

0.88 

0.98 

-0.10 

27 

88M 

201 

182 

19 

0.90 

3.86 

3.50 

0.36 

28 

89D 

5 

4 

1 

0.87 

0.10 

0.08 

0.02 

29 

89B 

125 

100 

25 

0.80 

2.40 

1.92 

0.48 

30 

MD1 

50 

47 

4 

0.93 

0.96 

0.90 

0.06 

31 

92  F 

359 

378 

-19 

1.05 

6.90 

7.27 

-0.37 

32 

92G 

89 

91 

-2 

1.02 

1.71 

1.75 

-0.04 

33 

SL1 

225 

228 

-3 

1.01 

4.32 

4.38 

-0.06 

34 

INI 

18 

29 

-11 

1.60 

0.35 

0.56 

-0.21 

35 

HI  1 

38 

92 

-54 

2.43 

0.73 

1.77 

-1.04 

36 

98X 

6 

3 

3 

0.57 

0.12 

0.06 

0.06 

999 

Non-Acc 

1222 

1325 

-103 

1.08 

23.48 

25.47 

-1.99 

A-15 


Table  A-4.  Hold-Out  (Validation)  Sample  TOS Fit  Diagnostics  by  Subgroup 


Siibeoiip 

TOS 

Obs.  N 

Exp.  N 

Diff.  N 

Ratio  N 

Obs.  Pet. 

Exp.  Pet. 

Diff.  Pet. 

ALL 

3 

5765 

5626 

139 

0.98 

30.66 

29.92 

0.74 

4 

4S44 

5041 

-197 

1.04 

25.76 

26.81 

-1.05 

5 

2128 

2230 

-102 

1.05 

1 1.32 

11.86 

-0.54 

6 

859 

857 

2 

1.00 

4.57 

4.56 

0.01 

Non-Acc 

5207 

5049 

158 

0.97 

27.69 

26.85 

0.84 

MALE 

3 

5105 

4970 

134 

0.97 

33.43 

32.55 

0.88 

4 

3816 

4026 

-211 

1.06 

24.99 

26.37 

-1.38 

5 

1733 

1858 

-125 

1.07 

11.35 

12.17 

-0.82 

6 

691 

627 

64 

0.91 

4.52 

4.11 

0.42 

Non-Acc 

3926 

3788 

138 

0.96 

25.71 

24.81 

0.90 

FEMALE 

3 

660 

655 

5 

0.99 

18.69 

18.54 

0.15 

4 

1028 

1015 

13 

0.99 

29.11 

28.73 

0.38 

5 

395 

372 

23 

0.94 

11.18 

10.53 

0.65 

6 

168 

230 

-62 

1.37 

4.76 

6.51 

-1.75 

Non-Acc 

1281 

1261 

20 

0.98 

36.27 
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1 1 1 13+ IV 
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5 
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6 

82 
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1.08 

23.48 
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Appendix  B  reports  the  raised  bonus  cap’s  forecasted  impact  on  MOS  channeling  under  the  five 
market  expansion  conditions,  for  all  applicants  and  by  subgroup.  For  comparison  purposes, 
results  under  the  existing  cap  are  reported  under  “Baseline.”  Table  B-l  reports  the  forecasted 
number  (“N”)  and  percent  (“%”)  of  accessions  by  MOS,  grouped  by  incentive  level.  The 
percentages  reported  for  all  applicants  represent  their  share  of  an  MOS  relative  to  total 
accessions,  whereas  the  subgroup  percentages  represent  their  share  of  an  MOS  relative  to  the 
total  for  a  subgroup.  Table  B-2  reports  changes  in  accessions  by  MOS,  both  in  number  (“N  +/-”) 
and  expressed  as  percent  increase  or  decrease  (“%  +/-”),  relative  to  their  corresponding  baseline. 
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Table  B-l.  Forecasted  Number  of  Accessions  and  Percent  of  Total  by  Subgroup,  Incentive  Level,  MOS,  and  Market  Expansion  Condition 
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TMe  B-l.  Forecasted  Number  of  Accessions  and  Percent  of  Total  by  Subgroup,  Incentive  Level,  MOS,  and  Market  Expansion  Condition 


Table  B-l.  Forecasted  Number  of  Accessions  and  Percent  of  Total  by  Subgroup,  Incentive  Level,  MOS,  and  Market  Expansion  Condition 
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Table  B-2.  Forecasted  Change  in  Accessions  Relative  to  Baseline  by  Subgroup,  Incentive  Level,  MOS,  and  Market  Expansion  Condition 
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Tahle  B'2-  Forecasted  Change  in  Accessions  Relative  to  Baseline  by  Subgroup,  Incentive  Level,  MOS,  and  Market  Expansion  Condition 
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Table  B-2.  Forecasted  Change  in  Accessions  Relative  to  Baseline  by  Subgroup,  Incentive  Level,  MOS,  and  Market  Expansion  Condition 
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Appendix  C:  Forecasted  TOS  Channeling  Effects 


C-l 


Appendix  C  reports  the  raised  bonus  cap’s  forecasted  impact  on  TOS  channeling  under  the  five 
market  expansion  conditions,  for  all  applicants  and  by  subgroup.  For  comparison  purposes, 
results  under  the  existing  cap  are  reported  under  “Baseline.”  Within  each  subgroup  results  by 
TOS  are  grouped  by  incentive  level,  which  is  indicated  under  the  “Level”  column.  Overall,  the 
layout  and  information  presented  in  Tables  C-l  and  C-2  are  similar  to  corresponding  tables  in 
Appendix  B. 
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Table  C-l.  Forecasted  Number  of  Accessions  and  Percent  of  Total  by  Subgroup,  Incentive  Level,  TOS,  and  Market  Expansion  Condition 

_ Market  Expansion  (I-IIIA’s) 

Baseline  0%  1.1%  2.2%  5.4%  7.5% 

Subgroup  Level  TOS _ N _ % _ N _ % _ N _ % _ N _ % _ N _ % _ N _ 
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Table  C-2.  Forecasted  Change  in  Accessions  Relative  to  Baseline  by  Subgroup,  Incentive  Level,  TOS,  and  Market  Expansion  Condition 
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Appendix  D:  Forecasted  Changes  in  Bonus  Dollars  Awarded  by  MOS 
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Appendix  D  reports  the  forecasted  changes  in  the  average  and  total  bonus  dollars  awarded  (in 
thousands)  from  raising  the  bonus  cap  under  the  five  market  expansion  conditions  by  incentive 
level  and  MOS,  for  all  applicants  and  by  subgroup.  For  comparison  purposes,  results  under  the 
existing  cap  are  reported  under  “Baseline.”  Table  D-l  reports  the  average  bonus  dollars  awarded 
(“Avg  $”)  under  each  of  the  five  market  expansion  conditions  and  changes  in  this  amount, 
expressed  as  percent  increase  or  decrease  (“%  +/-”),  relative  to  the  baseline.  Table  D-2  reports 
changes  in  total  bonus  dollars  awarded  under  the  five  market  expansion  conditions,  both  in 
amount  awarded  (“$  +/-”)  and  expressed  as  percent  increase  or  decrease  (“%  +/-”),  relative  to 
baseline. 
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Table  D-l.  Forecasted  Changes  in  Average  Bonuses  Awarded  Relative  to  Baseline  by  Subgroup,  Incentive  Level,  MOS,  and  Market 
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Table  D-2.  Forecasted  Changes  in  Total  Bonuses  Awarded  (in  Thousands)  Relative  to  Baseline  by  Subgroup,  Incentive  Level,  MOS,  and 
Market  Expansion  Condition 
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Appendix  E:  Forecasted  Changes  in  Total  Bonus  Dollars  Awarded  by  TOS 
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Appendix  E  reports  the  forecasted  changes  in  the  total  bonus  dollars  awarded  (in  thousands) 
from  raising  the  bonus  cap  under  the  five  market  expansion  conditions  by  incentive  level  and 
TOS,  for  all  applicants  and  by  subgroup.  For  comparison  purposes,  results  under  the  existing  cap 
are  reported  under  “Baseline.”  All  tables  changes  in  total  bonus  dollars  awarded  under  the  five 
market  expansion  conditions,  both  in  amount  awarded  (“$  +/-”)  and  expressed  as  percent 
increase  or  decrease  (“%  +/-”),  relative  to  baseline. 
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Table  E-l.  Forecasted  Changes  in  Total  Bonuses  Awarded  (in  Thousands)  Relative  to  Baseline  by  Subgroup,  Incentive  Level,  TOS,  and 
Market  Expansion  Condition 
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Table  E-l.  Forecasted  Changes  in  Total  Bonuses  Awarded  (in  Thousands)  Relative  to  Baseline  by  Subgroup,  Incentive  Level,  TOS,  and 
Market  Expansion  Condition 
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Forecasted  Impact  of  Market  Expansion  -  Illustrative  Cases 

To  complement  the  MOS  channeling  effects  analyses  presented  in  the  paper,  we  extended 
the  analysis  to  include  the  impact  of  raising  the  bonus  cap  under  different  market  expansion 
conditions.  These  conditions  are  purely  illustrative. 

Since  applicants  of  average  or  lower  quality  (IIIBs  and  below)  are  not  expected  to  benefit 
from  raising  the  bonus  cap,  their  total  in  the  Army  applicant  pool  were  kept  constant  in  our 
market  expansion  simulations.  That  is,  only  the  I-IIIA  segment  of  the  applicant  pool  was 
assumed  to  expand  in  our  analysis.  The  illustrative  I-IIIA  market  expansion  rates  employed  in 
our  simulations  are  (a)  =  1.1%,  (b)  =  2.2%,  and  (c)  =  5.4%. 

To  operationalize  the  non-zero  market  expansion  in  the  simulations,  the  required  weights 
were  applied  to  the  computed  choice  probabilities  of  I-IIIA  applicants.  For  example,  when 
aggregating  the  choice  probabilities  under  the  2.2%  market  expansion  condition,  those  computed 
from  the  choice  set  for  I-IIIA  applicants  are  weighted  by  1.022,  while  probabilities  obtained  from 
IIIB  applicants  and  lower  were  assigned  unit  weights.  Thus,  the  predicted  enlistment  choices  of 
each  I-IIIA  applicant  counted  an  additional  2.2%  in  the  corresponding  aggregated  MOS/TOS 
percentages.  Employing  this  weighing  scheme  resulted  in  the  desired  percentage  increase  in  total 
I-IIIA  applicants  for  each  condition. 

Figure  F-l  summarizes  the  forecasted  impact  from  an  expanded  market  of  high  quality 
applicants,  specifically  I-IIIA’s,  on  accessions  by  MOS.24  Overall,  an  expanded  applicant  pool 
of  I-IIIA’s  has  the  potential  to: 

■  Further  increase  the  percent  of  I-IIIA’s  accessing  to  higher  priority  MOS.  Specifically, 
expanding  the  market  of  I-IIIA’s  by  2.2%  and  5.4%  over  the  current  applicant  pool  could 
increase  the  percent  of  I-IIIA’s  accessing  to  higher  priority  MOS,  on  average,  by  12.1% 
and  15.6%,  respectively.  Among  the  higher  priority  MOS,  the  gains  are  projected  to  be 
greatest  for  13F,  89D,  and  92F  — increases  of  16-28%  each. 

■  Mitigate  the  potentially  harmful  channeling  effects  on  lower  priority  MOS  to  be  expected 
from  raising  the  existing  bonus  cap  to  $40K.  As  observed  previously,  raising  the  cap  is 
projected  to  channel  I-IIIA’s  to  high  priority  MOS  at  the  expense  of  low  priority  MOS. 
Holding  all  other  factors  constant,  our  simulation  indicates  that  even  a  modest  expansion 
of  the  applicant  market  (an  increase  in  I-IIIA’s  between  2.2  and  5.4  percent)  carries  the 
potential  to  ameliorate  these  channeling  effects,  such  that  all  MOS  enjoy  a  gain  in  higher 
quality  accessions  relative  to  existing  conditions. 


24  For  a  complete  breakdown  of  results,  see  Appendix  B. 
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Figure  F-l.  Percent  Change  in  Accessions  (AFQT  Category  I-IIIA’s)  Relative  to  Baseline  Under 
1.1%,  2.2%,  and  5.4%  Market  Expansion 
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